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CHAPTER  0 


INTRODUCTION 

0*1  General  Introduction 

The  context  of  this  study  was  the  desire  for  a  better  understanding  of  a 
set  of  concepts  we  believe  important  for  the  theory  of  computation. 
Distinctions  like  "serial  vs.  parallel  computation,"  "local  vs.  global 
properties,"  "addressed  vs.  associative  memory,"  :terative  vs.  recursive 
algorithms,"  are  frequently  used  to  refer  to  these  concepts,  often  as  if 
they  were  well-defined  technical  terms  used  with  substantial  knowledge  about 
the  conditions  under  which  these  forms  of  computation  would  be  advantageous. 
But  despite  their  wide  currency  in  an  intuitive  form,  they  have  not  as  yet 

received  any  satisfactory  formal  definitions,  nor  are  they  at  all  well  under- 
stood  even  in  their  intuitive  forms. 

We  felt  that  our  inability  to  formulate  satisfactory  definitions  was  due 
mainly  to  the  unavailability  of  thoroughly  analysed  special  cases  that  could 
serve  as  models  for  thinking  about  the  broader  issues.  Good  theories  develop 
rarely  outside  of  the  context  of  well-understood  real  prdbiaas,  and  it  is 
perhaps  not  surprising  that  work  directed  sharply  toward  obtaining  an  "abstract 
theory  of  computation"-*,  g. ,  the  mathematical  developments  in  current  theories 
of  recursive  function,  automata,  formal  linguistics,  and  the  like— has  been 
disappointing  in  the  extent  of  its  practical  illumination,  despite  its  often 
elegant  mathematical  quality.  Accordingly,  we  have  become  engaged  in  a  number 
of  attempts  to  clarify  the  nature  of  computations  in  some  probisms  of 
independent  interest.  In  the  present  study  w*  explore  the  properties  of  the 


simplest  class  of  automata  we  know  that  have  no  loops  or  feedback  but  are 
nevertheless  capable  of  some  non-trivial  computations.  Fortunately  they  are 
also  rich  enough  to  be  the  object  of  an  interesting  mathematical  theory. 

live- mathematical  results  described  :herein  permit  analysis,  to  a  certain 
level,  of  the  range  and  limitations  of  a  class  of  computing  machine  that  have 
been  widely  investigated,  by  empirical  methods,  for  possible  use  on  problems 
of  pattern  recognition. 

The  characteristic  feature  of  these  machines  is  that  'they  make  their 
decisionsr-about  whether  or  not  a  certain  event  fits  a  certain  "pattern" —by 
"adding  up"  evidence  obtained  from- many  separate  small  experiments.  This  is  a 
very  important  concept  because  it  is  so  clear  and  simple.  Most,  and  perhaps 
all,  more  complicated  machines  for  making  decisions  will  have  a  little  of 
this  character.  In  any  case,  until  we  understand  this  simple  concept  very 
thoroughly,  we  certainly  can  expect  trouble  with  more  advanced  ideas. 

Generally,  in  Science  and  Mathematics,  one  advances  by  understanding  first 
the  "linear"  systems,  and  these  machines  are  our  candidate  for  the  "linear  case 
of  the  parallel  machine  in  general."  We  will  bring  forward  a  number  of 
arguments,  at  various  points  in  the  teixt  to  support  this  view  (which  is 
a  methodological  position  rather  than  a  technical  matter). 

These  devices,  defined  below  in  §0.3  and  §1.2  are  most  fittingly  known 
as  perceptrons  in  recognition  of  Rosenblatt’s  contributions  toward  formulating 
mathematically  clear  definitions.  Under  this  name,  the  machines  have  been 
widely  investigated,  but  with,  generally  inconclusive  and  puzzling  results. 

The  empirical  tests  have  been,  by  and  large,  unconstrained  by  theoretical 
analyses  of  the  machines’  limitations:  such  analysis  as  was  attempted  was 


committed  chiefly  along  pertain  statiitical  directions  that  failed  to  shed 
much  light  on  the  relation  between  the  structures  of  the  ••patterns”  and  the 
ability  of  perceptrons  to  "recognise"  those  patterns. 

Our  theory  does  not  completely  characterise  these  limits.  For,  any 
such  theory  must  mediate  between  some  a  priori  classification  of  the  patterns 
themselves,  and  their  recognition  by  perceptrons.  It  would  be  too  much  to  ask 
for  an  absolute  classification  of  "patterns,"  for  this  depends  ultimately 
on  what  is  one's  goal,  i.e.,  what  one  is  interested  in.  We  have  chosen  to 
study  some  patterns  that  are  definable  in  terms  of  familiar  geometric 
concepts,  for  these  are  both  of  great  practical  interest,  and  are  profoundly 
well  understood  mathematically.  For  each  class  of  geometric  patterns,  we 
have  to  attack  the  problem  of  what  conditions  must  be  met  if  a  perceptron 
is  to  make  an  appropriate  recognition.  To  do  this  we  need  to  develop 
— ?.a.fyfcfc  tools,  and  often  new  ones  for  each  new  problem. 

Our  experience  has  been  that  such  problems  are  by  no  means  trivial. 

Some  of  them  baffled  us  for  a  long  time  before  we  found  suitable  analytic 
concepts  for  treating  them.  Some  of  them  led  to  solutions  quite  the  opposite 
of  our  intuitive  expectation.  Above  all,  we  were  repeatedly  surprised  at  the 
curious,  and  various,  mathematical  paths  ws  were  ledr-or  rather,  forced— 
along.  We  have  made  some  attempt  to  leave  traces  of  these  paths  (thus  running 
against  today's  mathematical  style  of  covering  completely  one's  intellectual 
tracks)  and  we  hope  the  reader  will  try  to  share  this  by  reading  the  book  -  re 
as  a  novel  in  which  characters  develop  and  interact,  than  as  a  sequence  of 
theorems  and  proofs. 


O.Z  Local  froi 


One  of  th«  moat  powerful  theme.  In  cybernetic  discussions  of  pattern 
recognition  i.  relate  to  tha  discovery  that  a.-ingiy  complex  "pattern."  can 
often  be  charact.ri.ad  or  ganaratad  by’locai"  process..*.  Tha  following 
examples  are  ia«a.it  to  indicate  tha  meaning  of  tha  quoted  word  a  and  to  ahov  why 

it  ia  difficult  to  find  a  fonaal  definition. 

Let  R  be  a  region  in  tha  ordinary  two-dinenalonai  Euclidean  plana.  Let 
X  be  a  finura  drawn  on  R.  e.g..  a  circle  or  a  pair  of  circle,  or  a  black  and 
uhite  .ketch  of  a  lean’,  face.  In  general  we  think  of  X  a.  nerely  a  auhaet 
of  the  point.  R.  Whan  we  talk  of  a  "pattern"  we  ueually  hare  in  nlnd  acme 
cun  of  figures,  e.g.,  all  circlea,  or  all  connected  figures,  or  ail  wiling 
face..  Wa  .hall  diacu.a  a  m-b.r  of  kind,  of  algorithm.  that  examine  figure. 

to  decide  whether  they  belong  to  a  given  class. 

To  talk  about  these  algsrittasa  we  will  have  to  introduce  sons  auxiliary 
concept..  Wa  adopt  the  word  ■■predicate"  for  any  function  al)  which  has  the 


value  1  for  sons  figures,  and  the  value  0  for  all  other  figures.  In  general, 
to  compute  <p(X)  wa  must  look  at  evary  point  of  R  to  check  whether  it  ia  in  X 
or  not.  In  some  special  case.  <j*X)  cm  be  computed  by  looking  only  at  a  proper 
subset  of  point.  R.  In  any  case  we  call  the  required  subset  of  R  the  jJWEl 
of«p.  Tha  simplest  kind  of  predicate  look,  only  at  a  single  point  of  R:  we 


*  a  a- 


“  0  if  not. 

IL7”  7>  i* th:  jo“  -  -  - — « .u,u  ^  P. 

n,  set  tfi  .  ,>}  cont.lntng  n  ^  there  >re  2,n 

77  ~  w  °£  *  -  *  -•  -  the  ^  of  the 

th  .  .  1  2  *""P»  the  IHttffi  of  a  predicate  is  „i»piy 

"1”to0B  *”  °f  ■“t«*  «  >*lch  it  really  depots. 

Predicates  of  finite  tupportar.  in  .  very  stron,  ^  ^  ^ 

8  "  t0  “ClUd'  geoaetric  interest.  However  »,  Bti 

put  t„i.  notion,  to  os.  in  Uefining  .  ve.k,r  Hot  „re  interesting  sens.  , 

^  ■“  bC81"  •“  *"  — «•  converity. 

-Hi^-££2v«Sitypredic«te  *  ,y\  ~ 


Fig.  0.2-1 

w,  say  thet  .  figure  X  i.  oonver  if.  given  .„y  palr  of  lts  ^  t„. 

— 116  U“  «Sl«iy.»itMn  X.  nu  is  true  of  each 

nos^f^'  2'"  h<“Ve  '"e  whole. ,«  ... 


Pn3  as  their  support,  but 
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figure  on  the  left.  Each  figure  on  the  right  he.  exceptions,  ee  indicated 

by  the  dotted  line..  Now  we  wish  to  compute  the  predicate  ^(Jt)  which 

has  value  1  if  X  is  a  convex  figure  of  the  plane  and  value  0  if  X  is  not 

convex.  Clearly  doe.  not  have  a  finite  support,  for  its  value  can 

depend  on  what  happens  anywhere  in  the  (infinite)  plane.  We  ask  instead:  is 

it  possible  to  find  a  collection  of  stapler  predicates  each  with  wall  support, 

together  with  some  staple  way  to  combine  thw  to  synthesise  t  t 

CONVEX 

To  b.  more  specific:  W.  .h.U  My  th.t  .  prsdlcnt,  *«)  t.  conlonoMv.l', 

IomI  if  there  1.  .  m-b.r  k  end  .  collection  ♦  (perhep.  infinite)  of  predicts, 

wno.e  support,  eech  conteln  no  note  then  k  points.  Furthenore  *  nust  hove 
the  property  that 


*(X)  -  1  if  and  only  if  <p(X)  -  0  for  every  sp  in  *. 


Ke  ask  whether  *C0||VEX  is  conjunctively  local.  (The  point  is  that  we  are 


ttvxttg  to  develop  ways  of  building  complicated  predicates  out  of  simple  ones.) 


The  answer  is  ytai  We  can  set  k  -  3  and  choose  some  predicate.  * 


xy* 


that  depend  each  on  only  three  points,  as  follows: 

Let  x,  y,  and  x  be  hny  three  distinct  points  that  lie,  in  that  order, 
along  any  straight  line  and  define 


X 


T~  — 


Vxy 2^  11  1  if  and  only  if 


p  is  in  X,  and 

<  *  is  in  X,  but 

ly  is  not  in  X; 

0  ^  any  other  case. 

””  0”ly  “y  *"  X  “»  ««Pe  having  „«)  .  .  f 
convex:  other*.,  there  «U  ^  9  U  by  b«°* 

»•  *-  X  hot  which  doe.  not  a,  ,  (C*U  “  '«•«»  **.  ends 

ii®  antir-ily  within  v  m. 

the  POl,!tS  ot  <*>*)  hot  In  X;  then  cp  .  ,  '  0036  ”  ^  °n'  ”f 

-  -e  i„  P*««ing  that  there  tH  »taple  fat-1 
♦convex  now:  w  can  writ.  '  Ibc 


vw<1, 

for  the  sum  of  any  collection  of  *eros  will  he 

makes  the  sum  at  least  unlty.  2er°’  WhUe  any  •“*«««  will 

Many  other  geometric  predicates  *r»  ^  . 

discussed  also  in  So.  4,1.  the  predicate  U”<!tlTelJ’  1°C*1'  *“th«  ««PU, 


♦ciRCLE®  •  l  <->X  I,  the  perimeter 

of  a  complete  circle. 


Theorem:  *CIRaE(X)  i.  conjunctively  local  with  k,  4 
the  Ptooi  i.h..M  c„  the  fact  that 
3  Str*l8ht  «-*.  determine  .  circle,  c 


points  x,y,*,  not  iu 


But  see  note  b.iow  for  the  degener.t 


e  case  of  2  or  fewer  points  in  X. 


•  /  * 
.  ■hi  ' 


A  '  -a 


■  « 


v  >  c  - 


Fig.  0.2-2 


NOW  obviou.1,  no  predict.  of  United  .upport  c.„  t.u  ^  ^ 

fUUr*  *  -  “*C“y  *  C1KU-  —  Po^t.  of  x  cy  he  outride  it, 

support  set.  But  if  X  is  not  a  circle  thin  ...  i 

nor  a  circl-  then  at  least  one  of  the  following 

two  kinds  of  events  must  happen: 


i)  there  .re  four  point.  *,y,x  end  .  in  x  which  do  not  lie  „„  the 


same  circle,  or 


it)  there  ere  three  point,  *„  end  .  i„  x  end  one  point  ...  «  in 


X,  which  do  lie  on  the  same  circle. 


To  Phi.,  chon.,  m  3  point,  v  v  ,o  in  x.  Ibey  ^ 

circle  C0.  if  (i)  i.  fele.  for  ali  point.  „  not  in  C().  then  .11  other  point, 
of  X  mu.t  lie  in  C()  end  w.  c.n  conclude  that  all  of  x  la  concined  in . , 

.UCU^.  now  if  (ii)  i,  fair,  for  all  points  w^,  in  Cq  thia  mean,  that  ' 
all  of  the  circle  r,Q  ^...contained,  in  X.  So  X  is  Cg,'  it  follows  that  * 
it  conjunctively  local  .With  order  r,"  i„. ..  „e  ^  ^ 

simultaneous  truth  of  a  lot  of  predicate,  each  with  *  4  support  point.. 


q.e.d. 
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NOTS:  if  x  contai  -only  0  or  1  or  2  points,  It  will  pass  the 

teat  fe>r  circle-ness.  There  U  absolutely  nothing  that 
can  be  done  to  prevent  this*  If  we  are  prepared  to 
ignore  0,  1,  and  2  point  figures,  or  to  conaider  them 
"degenerate  circle."  (which  is  hard  to  swallow  for  the 
2-ce.e)  then  *CIRCLE  is  conjunctively  local,  if  We 
must  reject  the  2-point  figures  as  non-circles,  then 
^CIRCLE  if  S2£.  conjunctively  local.  To  see  this,  we 
note  first  that  the  only  way  a  non-circular  X  can 
escape  as  defined  in  0.2,  is  by.  having  fewer 

than  3  points.  We  now  prove  that  there  is  no  way  to 
repair  this.  (This  is  interesting,  not  so  much  because 
of  the  theoren,  which  is  unimportant,  but  because  it  is 
a  simple  oxample  of  an  impossibility  proof.)  Suppose 
that  We  had  a  conjunctively  local  definition  for 

^CIRCLE '  i*e*»  a  "umber  k  and  a  set  <P  of  predicates 
each  of  support  s  k  for  which:  U 


X  is  a  circle  «->  a  (X)  -  0  for  all  a. 

U 


Then  let  X  consist  of  two  points  and  x^  Since  X 
isn't  a  circle  there  must  be  some  <p  for  which  q>  »  l. 

L«  *  •.  support  ..t  be  ?v  P2,  Pfc  «h.S  at 

least  Fy  ...,  Pk -are  not  in  X.  Then  <p  (x>  will  be  1  for 
«ny  other  set  X*  which,  contain.  xx  and^'  but  none  of 

p3»  Pk*  But  w*  c*n  always  find  such  sn  X'  which  is 
a  circle i  because  there  are  an  infinite  timber  of  circles 

through  x^and  Xy  and  vc  have  to  avoid  only  a  finite  set 
of  points. 

On  the  other  hand  certain  properties  are  essentially  not  conjunctivel; 
local:  e.g.,  the  property  of  being  connected.  (Set  \apter  5.)  As  ue 
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Him  1 1  understand  la  tor,  no  mllwr  mod  If  lent  Ion  of  that  property  make  it 

con lonct ivoty  local,  or  even  local  in  the  broader  sense  to  be  defined  in. 
the  next  section.  We  are  building  up  to  the  idea  -that  certain  properties  arc 
profoundly  global  in  that  separate  Local  observations  cannot  be  combined  in 
simple  ways  to  yield  conclusive  evidence  Cor  them. 

To  obtain  the  now  sense  of  local  Let  us  now  try  to  sepsrate  essential 
from  arbitrary  features  of  the  definition  of  conjunctive  localness.  The 
Intent  ion  Ls  clear:  to  divide  the  computation  of  a  predicate  $  into  two 


stages: 


Stage  l:  The  computation  of  many  properties  or  features 
which  are  easy  to  compute  either  because  they 
depend  only  on  a  sms'  1  subset  of  the  whole  input 
space  R,  or  are  very  simple  In  some  other 
interesting  way. 

Stage  2:  A  decision  algorithm  which  expresses  f  as  a 

function  of  the  results  of  Stage  1  computations. 
For  the  exercise  to  be  meaningful  this  decision 
function  must  also  be  particularly  homogeneous,  or 
easy  to  program,  or  easy  to  compute. 


1 


Sig.  0.2-3 

The  perticuLr  wey  In  which  thl.  intention  we.  reelieed  t„  our 
i.  extroaely  etblttery.  For  St.ge  1  we  might,  inate.d  of  r.etricting  the 
nomber  of  polnte  in  the  euppott  eete  of  the  locr.1  functions,  he,e  restricted, 
for  example,  their  diameter  (as  in  Chapter  8). 

For  Stage  2  there  are  any  number  of  candidates  to  replace  unanimity  as 
the  decision  criterion,  with  greater  claim  to  generality  and  very  little  loss 
in  computational  simplicity.  A  general  theory  would  have  to  undertake  the 
difficult  task  of  characterizing  the  complexity  of  all  possible  algorithms. 
Without  such  a  characterization*  the  requirement  of  Stage  2  must  retain  a 
heuristic  character  that  makes  formal  definition  difficult. 

In  this  study  ve  shall  confine  attention  to  a  class  of  decision  functions 
that  includes  unanimous  decision  as  a  particular  case:  that  is,  the  definition 
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jn  in  the  next  section  can  be  thought  of  as  derived  from  the 


above  scheme  by  replacing  unanimity  by  majority  or,  more  generally,  weighted 


vo  tint 


°-3  I’erccptrons— Definition 


Let  *  be  a  set  of  predicates.  We  say  the  predicate  *  is  linear  in  the 
set  *  if  it  can  be  expressed  in  the  formt*(X)  -  1  if  and  only  if 


E  a  q)(X)  a  0 

(pi*  V 


where  the  "coefficients"  a  and  the  "threshold"  G  are  real  number..  The 


unanimity  condition  used  in  the  previous  section  to  define  conjunctive 


localness  can  be  expressed  in  this  form  by  letting  -  l  for  all  <p, 

and  Q  -  0,  provided  we  do  not  mind  the  sum  becoming  infinite.  For 


“  °<jj  -  L  <p(X)  S  0  i«  true  exactly  when  no  <p(X)  ■  i.  So 


4'(X)  ~  1  jf  and  only  if  <p(X)  -  0  for  all  qi«*. 


Fig.  0.3 


K  ■' 


/ 


Ihe  possibility  of  Infinite  ewe  lends  to  more  fussy  amplications  In 
proofs  than  It  Is  north.  To  prevent  It  happening  we  shall  .Wls«»  the 

plane  hy  ...umihg  It  to  be  made  up  of  discrete  little  cares.  This  l. 
equivalent  In  effect  to  identifying  «*„«.  ^  dl£f„  „y  ^  ^ 

a~e  "tolerahci;."  Moreover  »,  .hall  consider  only  boused  figure.  X.  and 

hoose  .  so  that,  for  a  given  X.  only  a  finite  number  of  <p’s  make  cp(X)  «  1. 

With  these  stipulation,  (vhlch  will  be  set  our  core  carefully  i.ter)  ve 
define: 

A  pare,  ptron  1.  a  dglce  capable  of  comnutln.  . . 

~  Ull“r  ln  ^  alVtt  Mt  °f  *  ”f  "partial  -adi-.—  We  obtUn 
■^lie^  of  by  Imposing  restrictions  on  the  members  of  $. 

The  following  families  scan  to  be  particularly  interesting: 

(a)  Diameter-limited  per cent rnn«-  the  support  sets  of 
members  of  i  are  restricted  not  to  exceed  a  fixed 
diameter  ic  the  ordinary  metric  of  the  plane. 

(b)  Qgder-reatricted  perceotrnn.-  we  8ay  that  a  perceptron 

has  ^nifno  member  of  i  has  more  than  n  points  in  its 
support. 


^  Gss»bs  perceptrons:  the  members  of  $  have  unrestricted 
support  but  must  be"linaar  threshold  functions"  (i.e., 

the^-sl^ndVl"  t  he^thr  e  sho  Id"  0? '  *  *  ^  £r6ely  thelr  w^ts, 


*  *  --  %.•****>»&■  I-  - - 
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h*vn  order  1).  This  Is  equivalent  to  saying  that  each  <p 

in  4  Is  defined  by  a  signed  measure  on  R,  and  a  threshold 
9  . 

<d>  Random  perceptrona:  These  arc  the  form  most  extensively 
studied  by  Rosenblatt's  group:  they  are  order-restricted 
and  4  Is  generated  by  a  stochastic  process  according 
to  an  assigned  distribution  function. 

To  give  a  preview  of  the  kind  of  results  we  will  obtain,  we  present 
here  a  simple  example  of  a  negative  result: 


Thy rem  8,2.3;  A  diameter-limited  jctfceptron  cannot  determine  whether  or  not 
of  «  geometric  figure  are  connected  to  one  another.  The  proof 
requires  us  to  consider  Just  four  figures: 


'is  . 


Suppose  that  a  perception  could  distinguish  the  disconnected  figures 
Xq0  andX^  from  the  connected  figures  X^Q  andX^,  i.e. ,  by  whether  or 


E  <p  >  6 

tp  t 

that  is i  according  to  whether  or  not 


E  a  ip,  ii  ol,  <p  +  L  a 

group  1  ^  group  2  ^  group  3 


cp  +  E  a  tp  >  9 


Then  for  3^0  the  stsa  of  the  three  E'a  is  negative.  In  changing  to 

Xin  only  £  is  affected,  and  its  value  oust  Increase  enough  to  make 
.U  group  1 


the  total 


exceed  9.  If  we  change  X-q  to  Xq.  similarly  £ 

group  2 


must  increase.  But  if  we  change  X^q  to  X^  then,  both  2  and 

____  group  1 

£  will  have  these  increases;  E  is  unchanged  i-  every  case,  so 
group  2  group  3 


the  full  increase  must  be  even  more  on  the  positive  side,  and  the 


perceptron  must  accept X^  as  connected! 


Q.  E.  D. 


0;4  Seductive  Aspects,  of  Perceptrons,  I: 


Homogeneous  Pro 


and  Learni 


The  purest  vision  of  the  perception  as  a  pattern-recognizing  device 


is  the  following: 
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The  machine  is  built  with  a  fixed  set  of  computing 

elements  for  the  partial  functions  9,  usually 

obtained'  by  a  random  process.  To  make  it  recognise 

a  particular  pattern  (set  of  input  figures)  one 

merely  has  to  set  the  parameters  to  suitable 

values.  Thus  "programming0  takes  on  a  pleasingly 

homogeneous  form.  Moreover  since  "programs"  are 

representable  as  points  (c^,  ...  a^)  in  an 

n-diaensional  space,  they  inherit  a  metric  which 

makes  it  easy  to  imagine  a  kind  of  automatic 

prograssning  which  people  haye  been  tempted  to  call 

learning;  by  attaching  feedback  devices  to  the 

parameter  controls  they  propose  to  "program"  the 

machine  by  providing  it  with  a  sequence  of  input 

patterns  and  an  "error  signal"  which  will  cause 

the  parameters  to  change  in  the  right  direction 

when  the  machine  makes  an  inappropriate  decision. 

£ 

The  perceptron  convergence  theormaa  define 
conditions  under  which  this  procedure  is 
guaranteed  to  find,  eventually,  a  correct  set  of 
values. 

To  separate  reality  from  wiahful  -  thinking,  we  begin  by 
making  a  number  of  distinctions.  Let  4  be  the  set  of  partial 
predicates  of  the  perceptron  and  L(*)  the  set  cf  predicates 
linear  in  4.  Thus  L(4)  is  the  repertoire  of  the  perceptron  -the  set  of 
predicates  it  can  compute  as  the  parameters  range  over  all  possible 


*  See  chapter  10. 
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values.  Of  course  L(*)  could  In  principle  be  the  set  of  ell  predicates 
(on  2  ):  but  this  is  universally  recognised  as  being  impossible  in  practice, 
since  i  would  have  to  be  astronomically  large.  So  any  real  perceptron 
has  a  limited  repertoire.  The  ease  and  uniformity  of  programing  have 
been  bought  at  a  cost.  We  contend  that  the  traditional  investigations 
of  perceptrons  do  not  realistically  measure  this  cost,  in  particular  they 
neglect  the  following  crucial  points: 

<  i.  The  translation  of  geometric  patterns  or  predicates  on  the 

input  plane  R  into  n-dimensional  vectors  (o^  . . .  aR)  loses  the  geometric 
.  individuality  of  the  patterns  and  has  only  led  to  a  theory  which  can  do 

t  Uttl®  "°re  than  £2HS£.  number  of  predicates  in  L(*)!  As  a  result  not  many 

{  people  seem  to  have  observed  or  suspected  that  there  might  be  particular 

geometrically  meaningful  and  intuitively  simple  predicates  which  belong 
to  no  practically  realisable  set  L(*).  We  have  already  given  an  example 
of  this  for  the  diameter-limited  case  and  will  later  extend  it  to  the  order- 
limited  cases.  At  the  same  time  we  shall  show  that  certain  predicates 
which  might  intuitively  seem  to  be  difficult  for  these  devices  can,  in 
fact,  be  recognized  by  low-order  perceptrons. 

ii.  Little  attention  is  paid  to  the  size,  or  more  precisely,  the 
I  information  content,  of  the  parameters  cy  ...,  aR.  We  shall  give  examples 

!  (which  we  conjecture  to  be  typical  rather  than  exceptional)  where  the  ratio 

i  of  the  lar8est  to  the  smallest  of  the  coefficients  is  meaninglessly  big. 

I  Under  these  conditions  it  is  of  no  (practical)  avail  that  a  predicate  be 

!  c 


% 


In  L(*).  In  some  «...  the  information  capacity  needed  to  store  * 

is  greater  than  that  needed  to  store  the  whole  class  of  figures  in  the 
pattern! 

iii.  Closely  related  to  the  previous  point  is  the  problem  of 
t ime-o  f -convergence  in  a  "learning"  process.  Practical  perceptron.  are 
essentially  finite-state  devices.  It  is  therefore  vacuous  to  cite  • 
perceptron  convergence  theorem  (see  Chapter  10)  .«  a..urance  that  . 
perceptron  will  eventually  find  a  correct  setting  of  its  parameters  (if 
one  exists).  It  could  do  so  trivially  by  cycling  through  all  its  states, 
e.g.  by  trying  all  coefficient  assignments.  The  significant  question  io; 
how  fast  the  perceptron  converge.  relative  to  the  time  taken  by  this 
homecstat -like  random  procedure?  It  will  be  seen  that  there  are 
situations  of  some  geometric  interest  for  which  the 

convergence  time  can  be  shown  to  increase  more  than  exponentially  vUb 
the  sire  of  the  set  R. 

Perceptron  theorists  are  not  alone  in  neglecting  these  precautions. 

A  persusl  of  any  typical  collection  of  papers  on  "self-improving"  systmes 
will  provide  a  generous  sample  of  schemes  for  "learning"  0r  "adaptive" 

machines  which  lack  even  the  degree  of  rigor  and  formal  definition  to  be 
found  in  the  literature  on  perceptrons.  The  proponents  of  these  schemes 
never  provide  any  analysis  of  the  rang,  of  behavior  which  can  be  learned, 
nor  show  any  awareness  of  the  price  paid  to  make  learning  easy,  by 
restricting  this  range  with  hidden  assumptions  about  the  envirowaent  in 


I 
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device  is  to  operate.  One  is  tempted  to  detect  a  my.tique  of 
unintelligibiUty..  the  murkier  the  mechanism  the  greater  tKc  virtue,  .. 

it  vote  better  to  try  to  find  u„.„.ly,.ble  ,Mlogue,  f„  biologlcal 

nyateme  then  to  try  to  find  mechanistic  explanations  for  them. 

These  critical  remarks  must  not  be 

use  not  be  read  as  suggestions  that  we  are 

opposed  to  making  machines  than  can  "learn  "  i?wm  *u 

xearn.  Exactly  the  contrary.  But 

ve  do  believe  that  significant  learning  .t  .  significant  r.te  pre„,ppo>es 
non .  significant  prior  atructnre.  Simple  learning  scheme,  ^  on 
ad  just  ing  coefficient,  can  indeed  be  practical  and  valuable  .hen  the 
partial  function,  are  cldsely  matched  to  the  task.  „  they  are  in 
Samuel.,  checker  player.  A  p.rceptron  with  a  set  of  partial  function. 

properly  designed  for  a  discrimination  Wd  to  be  of  suitably  l0.  order 

will  have  a  good  chance  to  improve  its  r 

P  8  1 erfonaance  adaptively.  Our  deep 

objection  is  to  the  concept  of  giving  .  high.order  ptoblm  # 

quasi-universal  perceptron  whose  partial  functions  have  not  been 
constructed  .ith  any  particular  task  ln  „tnd. 

It  may  be  argued  that  people  are  universal  learning  machine,  and  so  a 

nte.  example  to  this  theais.  But  our  brains  are  sufficiently  structured 
ro  be  prograasnable  in  .  Mr.  g,neril  ^  tban  ^  ^ 

car  Sjltu*  1.  sufficiently  structured  to  provide,  if  „  .ctual  prograa> 

“  least  a  rather  complex  aet  of  interactions  .hich  govern  the  course  of 

whatever  the  process  of  self-programming  may  be.  Moreover  it  take,  time 

*r  u.  to  become  univeraal  learners:  the  slo.  transition  fro.  infancy  to 
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Intellectual  maturity  is  rather  a  confirmation  of  the  thaals  chat  the  rate 
of  acquisition  of  new  cognitive  atructur.  (l.e.,  learning)  t.  ,  aenaliMv. 
function  of  the  level  of  existing  cognitive  structure. 

Seductive  Aspects  of  Perceptronar  TT- 

Parallel  Computation 

The  percaptron  was  conceived  as  a  parallel-operating  device  m  the 
Ph.««c,l  «»..  that  the  partial  predicate,  are  computed  atmultaneoualy. 

From  a  formal  point  of  view  the  .mportant  aapect  Is  that  they  are 
computed  independently  of  one  another.  The  price  paid  for  thl.  1.  that 
all  the  tpi  must  be  computed,  although  only  a  minute  fraction  may  in  fact  be 
relevant  to  the  final  decision.  The  total  mount  of  computation  may 
become  vastly  greater  than  that  which  would  be  carried  out  in  a  ..quant  1.1 
process  that  can  decide  what  next  to  compute,  conditionally  on  the  outcome 
of  earlier  computation.  Thus  the  choice  between  parallel  and  serial 
methods  in  any  particular  altuation  must  be  based  on  the  relative  value  of 
reducing  the  (total  elapsed)  time  agaln.t  the  cost  of  the  additional 

computation  Involved.  In  the  case  of  the  perceptron  the  concept  of  order 
provides  a  basis  for  the  estimation  of  the  latter  quantltea. 

Even  low  order  predicates  may  Involve  large  amount,  of  waateful 
computation  of  information  which  would  be  Irrelevant  to  a  serial  computation. 
But  the  numbers  can  remain  within  physically  realisable  bounds,  especially 
if  a  large  tolerance  (or  "Uur..,  ls  acceptable.  High  order  predicate. 
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create  a  completely  different  situation.  An  instructive  example  is 

provided  by  the  essentially  global  predicate  of  connectivity: 

4  (X)  *  1  if  and  only  if  X  is  a  connected  figure.  As  shown  in  Chapter  5 

Tcon 

a  perceptron  for  this  predicate  or.  a  100  X  100  toroidal  retina  would  need 
partial  functions  that  look  at  (a  most  conservative  minimum  of)  more  than 
800  points.  In  this  case  the  computation  of  lo/  \1  functions  is  irrele¬ 
vant  to  a  perceptron-like  linear  threshold  decision:',  the  partial  functions 
are  themselves  global.  Moreover,  the  number  of  possible  partial  functions 
with  such  large  support  makes  nonsense  of  any  hope  that  a  realisable 
randomly-generated  set  of  them  would  be  sufficiently  dense  to  span  the 
appropriate  space  of  functions.  To  make  this  point  even  sharper  we  shall 
show  that  for  certain  predicates  and  classes  of  partial  functions,  the 
number  of  partial  functions  would  exceed  physically  realizable  limits  even 
for  a  perceptron  designed  specifically  for  the  particular  predicate. 

The  general  conclusion  to  be  drawn  is  that  the  appraisal  of  any 
particular  scheme  of  parallel  computation  cannot  be  undertaken  rationally 
without  a  theory  of  the  corresponding/dithotomy  of  problems  as  local  and 
global.  The  lack  of  a  general  theory  of  what  is  global  and  local  is  no 
excuse  for  avoiding  the  problem  in  particular  cases.  The  analyses  below 

*  Unless  the  predicates  are  specially  designed,  their  number  may  tu  n  out 
to  be  of  the  order  of  21000,  and  so  may  their  coefficients.  This  would 
make  it  necessary  to  compute  serially,  anyway,  since  all  the  power  of 
Niagara  Falls  would  not  be  enough  to  run  them  in  parallel. 


show  chat  it  is  not  impossibly  difficult  tt  develop  such  a  theory  for 
a  limited  class  of  computing  devices  such  as  the  perceptrons. 


Seductive  Aspects  of  Perceptrone,  III: 

The, Use  of.  Simple  Analogue  Devices 

An  attractive  feature  of  the  pcrcoptron  is  the  idea  that  the  linear 
threshhld  decision  function  can  be  computed  by  a,  very  simple  analogue 
device.  It  is  perhaps  generally  appreciated  that  the  utility  of  the 
scheme  is  limited  by  the  sparseness  o*  linear  threshold  functions  U  ,ie 
set  of  all  logical  functions.  However,  almost  no  attention  has  been 
paid  to  the  possibility  that  the  aer  of  linear  functions  which  are 
Eg£tically  realizable  may  be  rarer  still.  To  illustrate  this  problem  we 
shall  compute  the  minimal  ratio  between  largest  and  smallest  coefficients 
in  the  linear  representations  of  certain  predicates.  It  will  become 
apparent  that  this  ratio  can  increase  fanner  than  exponentially  with  the 
number  of  distinguishable  points  in  R.  It  follows  that  for  "big*'  input 
set8--say  larger  than  20--no  simple  analogue  storage  device  can  be  made 
with  enough  information  capacity  to  store  the  whole  range  of  coefficients.' 

To  avoid  misunderstanding  perhaps  wc  should  repeat  the  qualification 
made  in  connection  with  our  critique  of  the  perceptron  as  a  model  for 
“learning  devices."  We  have  no  doubt  that  analogue  devices  of  thi*  sort 
have  d  role  to  play  in  pattern  recognition.  But  we  do  not  see  the.  iny 
good  can  come  of  experiments  which  pay  no  attention  to  the  limiting  factors 


I 


which  will  assert  themselves  as  soon  is  the  small  model  is  scaled  up  to 

u  usable 

0.7  Mathematical  Plan:  Introduction  to  Part  I 

Part,  1  (Chapters  1-rIV)  contains  a  series  of  definitions  and  general 
theorems  required  for  Part, II.  It  would  be  difficult  to  struggle  through 
this  material  without  a  preliminary  picture  of  the  roles  these 
mathematical  devices  aredastlned  to  play.  We  can  give  such  a  picture 
by  outlining  how  we  will  prove  the  following  theorem: 

InSbreni  d.il  (Chapter  3);  Informal  V  -slop 

Suppose  the  retina  R  has  a  finite  number,  |r|,  of  points. 

Then  there  is  no  perceptron  (X)  >  0  that  will  tell 

us  whether  or  not  the  "number  of  points  in  X  is  odd  or  even" 
unless  at  least  tone  of  the  <p  '  s  has  support  *  |xj .  Thus 
no  bound  can  be  placed  on  the  orders  of  perccptrons  that 
solve  this  problem  for  arbitrarily  large  retinas. 

The  proof  uses  several  steps: 

S£®£_Ii  Jn  $1.1--$1»4. we  define  "perceptron,"  "order," 
etc.  more  precisely,  and  show  that  certain  details  of  the 
definitions  can  be  changed  without  serious  effects,  l.e. , 

that  2  a  2  6  can  always  be  replaced  (by  E  a  V  >  0. 

<f  ■  9 

Step  2.  in  }l,3_J»J _ _  define  the  particularly  simple 

cp- functions  called  "masks."  For  each  subset  A  of  the 
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rm„«  define  the  „.,k  ,.A(X)  t„  h.v.  value  1  if  th.  figure  * 
contain,  or  “cover,"  .11  of  A,  value  0  oUicr.1...  Ih.„ 
prov«  the  simple  but  important  theory  (<|  1.5)  that  if  a 
predicate  I, a,  order  *  k  («,{1.3)  In  any  ,.t  „f  <p. function., 
there  1,  an  equivalent  perception  that  u-e.  only  rca.k.  of 
support  s  k.  (Sea  §0.2.) 

Stc^  To  really  "get  at"  the  paritv-the  "odd-even" 
property— we  ask:  what  voarran«cmcnt«  of  the  input  apace  R 
loavcit  unaffected?  That  is,  we  ask  about  the  group  of 
transformations  that  have  no  effect  on  it.  This  seen*  like 
using  excessively  high-powered  mathematics,  but  it  se«na 
necessary  for  the  more  difficult  problems  so  we  should  get 
used  to  it  here.  In  this  case  the  group  is  the  whole 

permutation  group  on  R~«M>  set  of  all  rearrangments  of  it. 
points. 

~  In  Chapter  2  we  show  how  to  use  this  group  to 
reduce  its  pcrceptron  to  a  simple  form.  In  the  present 
case*  Rioup- invariance  theory  (,ee  section  2.2) 

*ows  that  for  the  parity  perceptron  all  masks  with  the 
same  aupport-aize-that  ia,  all  that  look  at  the  same 
numbers  of  (though  different  sets  of  )  points--can  be 
given  identical  coefficents.  Let  0^  be  the  weight 
assigned  to  masks  that  have  support-site  *  j. 
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Step  5.  It  then  follows  that  the  parity  perceptron  can 
be  written  in  the  form 


E  a  (W) >0 

o  J  j 

where  K  is  the  largest  support  and  (^)  i«  the  number  of 
subsets  of  X  that  have  j  elements.  (Define  |x|  to  be  the 
number  of  points  in  picture  X.) 

Step  6.  Because 


n  V  n(n  1)  •  •  •  (n  -  J +_U 
k  j  /  .  •  2  ♦  3  . . .  j 


which  is  a  polynomial  of  degree  j  in  n,  we  can  put  our 
predicate  in  the  form 

PR  (  |x|  )  >0 

where  P  is  a  pqlynoraial  in  |x)  of  algebraic  degree  *  K.  Now 
if  |x|  is  even,  PR(|x|)  >  0  while  if  |x|  is  odd,  PR(|x|)  *  0 
so  that  in  the  range  0  £  |x|  *  |r|,  Pr  must  change  its  direction 
JgJ  -  l  times.  But  a  polynomial  must  have  degree  ^  |  Rj  to  do 
that,  so  we  conclude  that  K  ^  |r|.  This  completes  the  proof 
exactly  as  done  in  Chapter  3.1. 


Thi.  .how.  how  the  algebra  work.  in.  For  some  of  th.  more  difficult 
connectedness  theorems  of  Chapter  5.  v.  need  somewhat  r^re  algabra 
and  group  theory.  In  Chapter  4  va  puah  the  idaaa 
about  the  geometry  of  algebraic  degree.  «  little  further  to  .how  that'  aome 
surprisingly  simple  predicate,  require  unbounded-order  perceptrons. 

To  see  some  simpler,  but  .till  charact.ri.tic  result.,  the  reader  might 
turn  directly  to  Chapter  8,  which  i.  almost  self-contained  bocau.e  it  does 
not  need  the  algebraic  theory. 


In  this  section  we  develop  the  theory  o£  the  linear  representation 
of  predicates  lined  on  an  abstract  set  R,  without  any  additional 
mathematical  structure.  The  theorems  proved  here  will  be  applied  in  later 
sections  to  sets  with  geometrical  or  topological  structures. 

Our  theory  deals  with  predicates  defined  on  subsets  of  a  given  base 
space  which  we  shall  consiatently  denote  by  R.  We  use  the  following 
rotational  conventions: 

1.1  Conventions 

(I)  Let  R  be  an  arbitrary  set  and  F  a  family  of  subsets  of  R. 

Using  the  letters  A,B,C,...,X,Y,Z  for  subsets  of  R  it  is  natural  to 
associate  with  F  a  predicate  <J^(X)  which  Is  TRUK  If  and  only  If  X«F. 

(II)  We  shall  use  the  letters  9  and  f  to  denote  predicates  defined 
on  the  set  of  subsets  of  R. 

We  shall  use  the  notation  f(X)  sometimes  to  mean  the  predicate  whose 
value  for  a  given  X  is  Tig  or  FAJjSE,  sometimes  to  mean  ah  binary  set 
function  whose  value  Is  1  or  0.  When  we  wish  to  employ  the  two  senses 
In  the  same  context  we  adopt  the  notation  F>(X)1  for  the  binary  function 
whose  value  Is  1  if  t(X)  is  TRJg  and  0  If  t(X)  is  FALSE.  We  will  usually 
use  this  only  when  there  Is  a  possibility  of  ambiguity,  e.g.,  to  distinguish 
between  [3  <  5]  -  1,  which  is  true  and  3  <  f5  »  which  is  false. 

(ill)  Occasionally  it  will  be  convenient  in  examples  to  use  the 


traditional  representation  of  t(X)  as  a  function  of  n  "boolean  variables" 
where  n  *  |r]  .  If-  the  elements  of  R  are  x^,...,xn>  it  is  traditional  to 
think  of  a  subset  X  of  R  as  an  assignation  of  the  values  1  or  0  to  the  x^ 
according,  to  whether  the  point  Xj^  is  in  X  or  not,  i.e. ,  "x^"  is  used 
ambiguously  to  stand  for  the  1th  point  in  the  given  enumeration  of  R, 
and  for  the  set  function  |x1eX‘f.  This  notation  is  particularly  convenient 
when  +  Is  represented  in  the  form  of  a  standard  boolean  function  of  two 
variables.  Thus  x^v  x^  is  a  way  of  writing  the  set  function 


cp(X)  «  fi^ex  or  x^exl. 


(iv)  We  need  to  express  the  idea  that  a  function  may  depend  only  on 

* 

a  subset  of  the  points  of  R.  We  denote  by  S(cp)  the  smallest  subset  S  of  R 
with  the  property  that,  for  every  subset  X, 


9(X)  -  cpOC^S)  • 


We  call  S(«p)  the  support  of  <p. 

1.2  Functions  Linear  with  Respect  to  a  Class  of  Predicates 

(v)  Let  $  be  a  set  of  binary  set-functions  on  R.  We  say  that  Jr  is 
is  a  linear  threshold  function  with  respect  to  i  If  to  each  number  <p  of  ♦ 
there  correspond^  a  real  number  such  that,  for  some  real  number  8; 

•§C 

For  an  infinite  space  R,  some  predicates  will  have  S(cp)  undefined.  For 
example,  suppose  that  <p(X)  *  1  <■■>  {X  contains  an  infinite  set  of  points}. 
For  then  one  can  determine  <p(X)  by  examining  the  intersection  of  X  with  any 
set  of  S  that  is  the  complement  of  R  in  some  finite  set.  And  there  is  no 
minimal  such  S. 


S'-*.--  a* 


too  -  r  S  q_9(X)  >  A 

tp«*  Y 


This  can  be  written  more  simply  as 


|  -  Te  >  0"1- 

We  denote  by  L<*)  the  set  of  functions  t  expressible  in  this  way. 

Proposition:  The  following  formal  modifications  lead  to  equivalent 

definitions: 

j£  $  assumed  to  contain  a  constant  function,  6  can  be  taken 
to  be  zero. 

(2)  The  inequality  sign  ,,<5'  can  be  replaced  by  ">»"  ' 

(3)  It  can  be  assumed  that  the  exact  equality  ®  never  arises. 

(4)  we  can  restrict  the  a*  «.  be  rationale  or  integers. 

(5)  The  choices  allowed  under  <1>~<4>  can  be  made  independe,*  iy. 
Proof:  Most  points  are  obvious.  To  prove  (3)  for  general  real 

coefficients  note  that  there  are  countably  many  X’s  and  so  countably 
many  values  of  Sy  To  prove  (4),  for  integer  note  that  we  could 

make  all  the  a* s  even  and  put  8  »  1. 

Our  most  consnon  choice  will  be  to  take  the  a±  as  integer  and  0  as 

zero. 

onjjoatlaa:  In  vl«  of  thl.  definitional  lnverlance.  It 
Bight  he  useful  to  ehbrevlete  the  ^*00  >  elnoutlon  to  elnply 

*  But  some  don't  hold  In  the  lnflnlte-retlna  eeae. 


t 


w 


in  view  of  option  3.  Then  one  could  go  on  end  uae  inner-product  notations 
like  or  even  However,  we  have  often  found  the  form  with 

explicit  0  more  convenient. 

ft 

1.3  The  Central  Concept  of  Order 

The  order  of  t  is  the  smallest  k  for  which  there  is  a  I  satisfying 


<pc#  — >  |  S(<p)  1  *  k 


il 


where  |s(<p)|  is  the  cardinality  of  S(<p). 

Functions  of  order  1  appear  in  the  literature  under  the  name  of 
"linear  threshold  functions."  Note  that  the  concept  of  order  would  be 
unaffected  by  imposing  the  condition  that  i  contain  a  constant  function. 

It  follows  that  it  would  not  be  changed  by  assuming  0  ■  0  in  the 
definition  of  L(t).  Clearly  neither  can  any  of  the  other  options  of 
1.2  affect  the  value  of  the  order  of  a  predicate. 

Definition:  <p  is  called  a  mask  if  there  is  a  set  A  such  that 


fp(X)  -  fx  O  A"]. 


We  denote  this  function  by  tp^. 

*  We  emphasize  that  the  order  of  a  predicate  is  defined  absolutely- -not 
simply  relative  to  a  particular  *-cla««. 
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In  point-function  notation  a  maak  <pA  ia  a  function  of  than  form: 

ylAy2  A . AV 

where  {yi}  is  the  subset  A  of  R. 

In  particular,  the  constant  function  <p(X)  »  l  i*  the  mask  with 
support  “  0. 

1*4  Examples  of  Linear  Representation 

Proposition:  All  aMsks  are  of  order  1. 

Proof:  For  each  x*A  define  <j>x(X)  as  fx*xl.  Then 

’’*  ■  r  s  I-,,  *  i*n  • 

xcA  1 

In  particular  the  individual  point-function  «p  and  <p  are  of  order  1. 

*  y 

Similarly  the  functions  xvy,  xA  y,  x3y  are  of  order  1.  But  the 
"exclusive  or,"x©y  and  its  complement,  x  «  y,  are  of  order  2. 

Example  (i) 

*1V*2VX3  is  of  order  1: 

rx1  +  x2  +  x3  >  Ol 
X1AX2AX3  is  also  of  order  1: 

Ixj  +  x2  +  x3  >  2~] 


j 


! 

? 


- 


«  * 


x  ~2  -  fXi  +  (i  -  xp  >  ll  “  -  Xj  >  0*^  is  of  order  1. 

*2  «i  -  r«2  +  a  -  xx)  >oi  -  r»2  -  >  -ii. whtch  is 

also  x2»  is  of  order  1. 

Example  (ii) 


,»  which  is 

X1X2V*1*2  "  r*lX2  +  "  xl)(1  "  X2*  > 

-  -  x^  -  x2  >  -ll 

is  of  ordsr  2.  (Proof  that  it  is  not  order  1  is 
in  Chapter  2.) 

Example  (ill) 

Let  M  be  an  integer  0  <  M  <  |r]  .  Then  the  "counting  function" 


♦“(x)  -  r(x|  - 


*  *. 

i- 

* 

i 


•i 


•»- 


n  : 

i. , 


1, 


i  ‘ 

» 


\ 


Which  recognises  when  X  contains  exactly  M  points,  is  of  order  2 
Proof:  consider  the  representation 


9M(X) 


f(2M  -  1) 


£  x.  + 
all  i 


(-2)  £  x  x  =  M2!- 

*h  1 J 


For  any  figure  X  there  will  be  Jxj  ten*  x  with  value  1,  and 

III  •  (III  -  1)  „ 

2  tarns  value  1*  Then  the  predicate  la  equal  to 

♦M(X)  «  f(2H  -  l).|xl  -  |x|-(jxj  -  1)  -  M2  *  0\  -  r<|x|  -  H)2  SO 


and  the  only  (Integer)  value  of  |x|  for  which  this  Is  true  is  |x|  «  M. 
Observe  that,  by  decreasing  the  constant  tens,  we  can  aodify  the  predicate 
to  accept  co'^nts  within  an  arbitrary  Interval  instead  of  a  single  value. 


3  5  7  IXI" 


Rjxi  -  5) 2  S4l  -  T3  S  jxl  s7l 
Fig.  1.4 


Note  that  the  linear  fona  for  the  counting  function  does  not 
contain  R  explicitly.  Hence  it  works  as  well  as  for  an  infinite  space  R. 

Q.E.D. 


/ 


7 


& 


!^,  - . 
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Exaaple  (iv) 

functlOM  r|x|  2 1,1  -  n«i  *  hi  ot  otder  t  bec.u.„ 

*re  r'Pr««t«l  by  r  S  xt  *  Ml  „ni  r  2  xt  s  K1. 

*  «.  in  P...i„g  thlt  „  c.„  obt.‘n  .tbltr>ry  fui)c.ion 

°f  *  fr»  predicat,,  by  „ltlng 


f(lx|)  -  f(0)  +  2  (f(k))*r|x|  i  k) 

k*l 


l;l 


i(0)  +  2  (f (k))  -  f(k  .  jj) 

k“l  ' 


This  fact  la  used  in  §8,2. 

lvS  ^  "Poattive  Normal  9nTm  - - „ 

The  order  of  a  function  can  be  defined  by  exaaining  its 

repre-  station  a.  a  linear  threahold  function  with  repaect  to  the  set  of 
n*»ks.  To  prove  this  we  first  show 


Theorem  1. 


feery  is  a  linear  threshold  function  with  re.pect  to  the 
set  of  all  masks. 

Any  Boolean  function  K*t . »n>  can  be  written  In  the  dl.  June  live 

normal  form 

♦(x)-c1(x)vc3(x)v...v/Cp(x) 


where  each  CjL(X)  has  the  form 

Zl/>  *2  A  ”•  A  zq 

and  each  2  is  either  an  x£  or  aa  And  since  at  most  one  of  the  C±(X) 
can  be  true  for  any  X,  we  can  rewrite  f  using  the  arithmetic  sum: 

«X)  *  C1(X)+C2(X)+  ...  CyCX) 

(even  for  infinite  suras).  The  bare  over  the  letters  can  be  eliminated 
by  using  the  equation 

"  a(1  “xj>P  -  op  -  coCjP 

where  a  and  p  are  conjunctions,  so  that  any  bar  on  a  torn  within  a 
conjunction  can  be  removed. 


1 


*1 


When  all  the  bars  have  been  eliminated  and  like  terms  have  been 
collected  together  we  have: 

t(X)  -  Sx^X) 

where  each  ^  is  a  mask,  and  each  a±  is  an  integer.  Since  u  <X)  is 
0  or  1,  (A)  is  equivalent  to 


w 


♦(*)  *  >  o  |.  (B) 

Saaii:  rx,  +  x2  +  x3  i.  cidl  -  xx  +  x2  +  x,  -  »* -a,x,,ai  +4X  x  . 

Theorem  l.S.2:  The  representation  (k\  is  unlone.  2  3  3  1  1  2  3 

To  see  this  let  fy)  be  a  set  of  masks  and  {^3  a  see  of  non-zero 
numbers,  choose  a  k  for  which  S<V  i,  „lntoli  there  t,  }  ^ 

that  SCcpj)  d  S(cf^) .  Then: 

9k(S(V)  *  1 


V«<V>  *  0 


j  i  k. 


It  follows  that  Sj^lX)  1.  not  Identically  zero  since  it  has  the  value 
ak  for  X  =  S(  «pk) . 

Now  if  Sa^X)  =  ^icpi<X)  for  ail  X,  then  E^-p^q^X)  «  0 

for  all  X.  It  follows  that  all  0j  -  This  prove,  the  uniqueness  of 
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a'°  l'0PreSC,'“C1°"  (A)  “htC"  "  -U  call  the  positive  o£  t 

Note  that  the  positive  aormal  fora.  he.  the  valves  0  end  1  ..  ordln.ry 

arithmetic  „„ms!.  i.e..  without  thef  1  device  of  interpreting  the  validity 
of  an  inequality  ns  n  predicate 

The°re.ii  _LJ_5^3 :  f  is  of  order  k  if  and  only  if  k  is  th*  1 1 

y  it  k  ts  the  smallest  number 

for  which  th fere  exists  n  set  ft  „f  masks  Satisfying 

*e*  ->  |s(cp)  |  5.  k 


I„  t  -  rS,i<Pi  >  01  each  ^  e»„  be  replaced  by  it.  po.ltlve  mimtl 

form.  If  |S(91)  |  <  k,  this  will  be  true  of  all  the  mask,  that  appear  ln 
the  positive  normal  form. 

Example: 

A  "Boolean  form"  has  order  no  higher  than  the  degree  in  it.  disjunctive 
normal  form.  Thus 


... x,x 


ijkVA  "  *ijkVj  •  SjkVj’ 


illustrating  how  the  negations  are  removed  without  raising  the  order.  This 

particular  order-3  form  appears  later  ($„  1„  a  perception  that  recognise, 
convex  figures. 


* 
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I£  *1  '>*■  »!  *'*  *2  K«>  order  Oj,  then  J  ,„d 

ti  H  have  order  £  0^  f)^. 

Proof:  tot  -  rS,A  >  Olaixl  tj  -r^  ,, j  >  ol.nd  that  th, 

coefficients  are  chosen  so  that  equality  never  erlaea.  Then 

h®k  m  f<sW(syV  >01, 

-  >  $ 

But 

lS<Vj>l  <  j  5(^)1  +  |S(¥j)|. 

The  other  conclusion  follows  from  •  \  „  fy#xT 

Example: 

Slnoo  ♦>«(*)  .  rr  X  a  Hi  «  Tx  =  Mil  „  conoludo  that  ♦"  ha. 

order  s  2,  the  result  of  example  (iii). 

Question:  What  can  be  said  about  the  orders  of  &  t2“|  and 
i2X?  The  answer  to  this  question  may  be  surprising,  in  view  of  the 
simple  result  of  the  previous  theorem:  It  is  shown  in }  4  that  for  any 
order  n,  there  exis  «  pair  of  predicates  ^  and  ^  both  of  order  1  for 
wJiich  (tj  a  i2)  and  <tj,  v  f2)  have  order  >  n.  In  f*ct,  gUppo?e  that 

»v-  AUjjuC  where  K,  B,  and  C  are  large  disjoint  subsets  of  R.  Then 

*t  "  r,X''AI  >  IVPlI  «nd  *2  -  nxA»l  >  lxnc|]  ,«ch  have  ordar  1 

because  they  are  represented  by 
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V*  Xi  '  V*  "  01  “  V»  **  •  **  >  01 

bUt  "  Sh‘U  *”  th*t  <*1  a  *2)  and  <#l  v  *2)  are  not  even  of 

in  the  sense  debited  lnJ  f.6  belou.  0„  ^  ^ 

one  can  he  sorptfsed  In  specie!  cases:  see,  e.s. .  yheorem^  5.5. 

1,6  Predicates  of  Finitg  Order 

Strictly,  a  predicate  is  defined  for  a  particular  set  R  tt 
fakes  no  forma i  sense  to  speak  of  the  name  predicts  fer  different  R's, 
However  the  motivation  of  „„r  work  was  entirely  from  -predicates"  defined 
independently  of  R-e.8„  the  number  of  events  in  the  set  X,  or  other 

Seomotric  properties  of  fibres  in  a  real  Euclidean  plane  to  which  X  and  R 

provide  mere  approximation*  «r_  v. 

PP  oxrmations.  To  be  very  precise  we  could  use  a  phrase 

-h  as  ISSiaSSjS*  to  refer  to  a  saner.!  construction  which  define. 

a  predicate  for  each  of  a  l.r8e  class,  of  sets  R.  In  8e„er.l  (except  m 

this  section,  we  shall  use  -predicate-  in  this  wider  sense. 

Suppose  we  are  siven.  predicate  scheme  ,  which  induce,  a  predicate 
*8  for  each  of  a  family  W  of  retinas.  „e  .hall  .ay  that  *  is  „f  ,inite 
order  if  the  orders  of  the  *R  are  uniformly  bounded  for  ,u  ^  ^ 
appropriate  family,  Jwo  e»mples  will  make  this  clearer: 

«)  Let  (Rl}  he  a  se,„ence  of  set.  with  |rI  .  t.  por  ,.=h  ^ 

is  a  predicate  defined  by  the  predicste  scheme  ♦“*(x>  which  asserts, 
for  XC  Rj,  that  "|x|  is  an  even  number."  As  we  will  ln 

31’  the  °rder  °£  8ny  "uch  *i  ^  I-  Thus  ♦*«  is  mt  of  finite 


•'V'  * 


order. 


tTEN‘ 

*tw  -  rut  -  10 1 

We  ehell  ehow  in  1.0  Chet  it.. 

H  cn*c  is  of  order  2  for  ail 

(obviou.iy)  of  order  for  *  .  _  ~  1 

f  *  finite  order,  w.  .,,U  ‘  7 

We  «H11  tey  in  thi.  cnee  that  it  i*of  order  2. 

In  these  cases  one  could  obtain  the  .a».  At  u 
irfinir  dichotomy  by  conidering 

«  «t«  R:  on  an  infinite  retina  the  predicate 

*mf*>  -  r  (x\  -  iol 


is  of  finite  order,  in  i 

’  xn  fact  of  order  -  2,  while 


WX>  -  fiXJ  i,  even] 


he.  no  order.  «•  eft„  look  et  probl..  ln  thl. 

"ill  diecu.,  foraeliution  of  ts  *  *"  S? 

melieetion  of  the  concept  of  en  infinite  perceptron.  ,t 

should  be  noted,  however  that  m, 

’  t,Wt  th*  "*•  °f  perceptron.  doe.  not 

cover  .1,  c....  For  exeaple  the  pradlcete 


meaningless 


is  well-defined  end  of  order  1  for  any  finite  R.  It  is 

for  infinite  R,  yet  we  would  llke  to  toneider  the  eorr.eponding 
predicate-scheme  to  have  finite  order. 


T 

i 

} 
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CHAPTER  2:  THE  CROUP  THEORY 


In  this  chapter  we  consider  linear  threshold  functions  that  are 
invariant  under  groups  of  transformations  of  the  points  of  the  base- space 
R.  The  purpose  of  this,  realised  finally  in  Chapter  V,  is  to  establish 
a  connection  between  the  geometry  of  R  and  the  question  of  when  a 
geometric  predicate  can  be  a  linear  threshold  function. 


2.1  Example:  Coefficients  Averaged  Over  A  S 


02331 


As  an  introduction  to  the  methods  introduced  in  this  section  we 
first  consider  a  simple,  almost  trivial  example.  Suppose  we  wish  to 
prove  that  the  function  v  x^  is  not  of  order  1.  To  do  so  we 
might  try  to  deduce  a  contradiction  from  the  hypothesis  that  numbers 
a,  p,  and  6  can  be  found  for  which 


'|f(x1,X2)  -  XjX^XjXj,  -  ocx1  +  p^  >  0. 


We  could  proceed  directly  by  writing  down  the  conditions  on  a  and  p: 


*1  »  °  x2  -  0  — >  0  >  0 

Xj, .1  *2rm  °  5"*>  a  *  9 

xr«  o  x2_«  1->M?  . 

x^  ■  1  x2  ■  l  — >  a  +  p  >  0 


In  this  simple  case  it  is  easy  enough  to  deduce  the  contradiction: 


S 


2 


« 


t 


1 


a  ^  6  and  9  <  0 
ft  £  0  and  0  <  0 


— >  a  <  0 

— >  p  <  C 


— >  a  +  p  <  a 


— >  a  +  p  <  0 


( 


t 

i 

? 


I 


while  by  hypothesis  a  +  p  >  0.  But  arguments  of  this  sort  are  hard  to 

it 

generalise  to  more  complex  situations  involving  many  variables.  On  the 
other  hand  the  following  argument,  though  it  may  be  considered  sore 
complicated  in  itself,  leads  to  elegant  generalisations.  First  observe 
that  the  value  of  t  is  invariant  under  permutation  of  x^  and  x2»  that  is, 

♦  (Xj^.Xj)  «  WXj.Xj). 

Thus 

axj^  +  P*2  >  0 
ax2  +  px^  >  0 

hence 

m  *i  -  m  »2  > e 

by  adding  the  inequalities. 

Similarly 

■0*1  +  P*2  ^  9 
0*2  +  P*L  *  9 

yields 

*  One  can  say  the  same  of  geometric  arguments  about  hyperplanes  separating 
vertices  of  the  n-dimansional  unit-cube. 


t 


1 


$ 


\ 

i 
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It  follows  that  if  we  write  y  for  a  ^  ,  then 


*(x1,x2)  -  fvx1  +  yx2  >  e| 


i.e. ,  we  can  assume  that  the  coefficients  of  x^  and  x2  in  the  linear 
representation  of  tjr  are  equal.  It  follows  that 

*(x)  -  [y|x|  >  •'}  -  Ty|x|  -  e  >ol 

(if  we  assume  that  the  apace  X  has  only  the  two  points  x^  and  xp. 

Now  consider  three  values  of  X, 

*0  -  A  |ig  -  o  vl*|  -  8  SO 

-  {x^  (Xjl  -  1  y|x[  -  e  >0 

*2  -  {x^Xjj}  jX^  -  2  Y|X|  -6^0 

Since  Xq  and  Xj,  satisfy  t»  and  X^  does  not,  the  first-degree  polynomial 

vlx|  -  9  in  Ixl  would  have  to  change  direction  twice,  from  positive  to 
negative  and  back  to  positive  as  Ixl  increases  from  0  to  2.  This  is 
clearly  impossible.  Thus  we  learn  something  about  ♦  by  averaging  it  over 
the  permutations  that  leave  it  invariant.  (The  method  is  similar  to  that 
used  in  Haar  Measure  theory.  See  2.5.  In  fact,  for  order  1,  it  is  the 
same  method.) 


« 


2.2  Equivalence-Classes  of  Predicate* 

The  generalization  of  this  procedure  Involves  consideration  of  groups 

of  transformations  on  the  set  R,  and  functions  if  invariant  under,  these 

groups.  In  anticipation  of  application  to  geometrical  problems,  wt  recrU 

the  mathematical  viewpoint  of  Felix  Klein:  every  interesting  -geometrical 

property  is  an  Invariant  of  some  transformation  group. 

Let  G  be  a  group  of  transformations  of  R  onto  itself.  If  g«G  and  X  C  R 

we  define  Xg,  the  result  of  transforming  X  by  g,  to  be  the  set  obtained  by 
applying  g  to  each  member  of  X: 

Xg  -  {y  lx«X  Ay  -  xg}. 


Then  we  can  define  ah  equivalence  relation  tp  E  «p'  of  predicates  with  respect 

G 

to  the  group  by 


<?>  -  9'  if  *nd  only  if,  (3g)(VX) (qKX)  -  p' (Xg)). 

G 

That  is ,  $  is  equivalent  to  if  there  is  a  transformation  g  such  that 
♦  (X)  and  ♦'(Xg)  are  always  the  sane.  Our  main  theorem  shows  that  If  a 
perceptron  is  to  classify  patterns  in  a  way  that  is  invariant  under  group 
G,  then  its  computation  can,  and  in  a  sense,  can  only  depend  on  the 
G-equi valence  classes  of  its  ♦-functions. 


2.3  The  Group  Invariance  Theorem 

Let  (i)  G  be  a  ffy^te  group  of  permutations  of  R 

(ii)  *  be  a  set  of  predicates  on  R  closed  under  G, 

i.e. ,  g«G  — >  where  we  define  <Pg  so  that 

<pg(X)  =  <f(Xg) . 


.1 


vs< 

*  3 

» 
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C 

<iti)  i  be  in  L(*)  and  invariant  under  C. 

Then  there  exists  a  linear  representation  of  t. 


*  -  r  £  a  <p  >  ol 

<pc*  * 

for  which  the  coefficients  depend  only  on  the  G~ equivalence  class  of 
cp,  i.e. , 


G 


Remarks: 

(1)  These  conditions  are  stronger  tfctn  they  need  be.  To  be  sure, 

the  theorem  is  not  true  in  general  for  Infinite  groups.  A  counterexample 
will  be  found  in^  11.  A.  However,  in  special  'cases  we  can  prove  the 
theorem  for  infinite  groups.  An  example  with  interesting  consequences 
will  be  discussed  later.  (See  {  11.2  .)  it  will  also  be  seen  that 

the  assumption  that  G  be  a  group  can  be  relaxed  slightly. 

(2)  We  have  not  investigated  relaxing  condition  (il)  ,  and  this,  would 
be  interesting.  However,  it  does  -ot  interfere  with  our  methods  for  showing 
certain  predicates  to  be  not  of  finite  order.  When  the  -theorem  is  applied 
to  show  that  a  particular  t  is  not  in  L(*)  for  a  particular  #,  it  shows 
also  that  t  is  not  even  in  the  possible  larger  L  of  *  closed  under  G.  If 
one  found  a  useful  notion  similar  to  but  more  delicate  than  "order,"  one 


C 


« 


night  h*ve  to  find  a  cot respond ingly  sharper  invariance  theorem  for  It. 
Proof:  Let  the  given  linear  representation  of  i  be: 


♦  -  T  E  a(tp)  9  >  Ol . 


The  new  representation  will  be: 


♦  -  Te  p(q>)  9  >  ol. 


where 


p(<p)  -  E  a(«pg). 
g«C 


It  is  clear  that  P(q>).  so  defined,  depends  only  on  the  equivalence  class 

of  9.  For  if  9  E  9' ,  theu  such  that  9'  ■  9^  and 
6 


;  ! 
o  ? 


p(9')  -  E  a<9’8)  -  E  a(9B()»)  •  E.  01(91)  -  P(9>- 

ttCG  CtC  nen  /ki/' 


It  is  equally  easy  to  see  that  ♦  is  indeed  given  by 


♦  -  Te  p(9)  9  >  ol 


Choose  any  X.  Suppose  that  t(X)  -  1.  Then  t(Xg)  •  1  for  all  g«G,  i.e., 


% 
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s  a(<p)  <p(Xg)  ■  S  a (<p)  <n(X)  >  0  for  all  gcG. 

tpc*  (pc* 


If  wc  substitute  *  *  for.  *g  this  can  be  written 


3  sOpg  *)  «p'  (X)  >  0  for  all  gcG 

9'  c* 


which  is  the  sane  as 


2  <*(%)  <£{X)  >  0  for  all  gcG. 

(pc* 


Adding  these  equations  tern  by  tern: 


S  2  a((pg)  qp<X>  >  0 
gcc  q>c* 


i.e.  i 


E  (  2  <P(X)  >  0 

tp«*  ^gcG  J 


1»  6#  * 


£  P<9)  <p(X)  >  0. 

tpc* 


Similarly,  if  *(X)  ^  0  we  obtain 


2  P(f)  <p(X)  s  0. 

(pc* 


1 


I  , 


* 


I  i 


|  I, 

'i 


3-; 


* 

* 


o 
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This  proves  the  theorem.  For  readers  to  whom  these  Ideas  seem  difficult 
to  work  with  abstractly,  some  concrete  examples  of  the  equivalence  classes 
will  be  useful;  the  geometric  "spectra"  of  $5.2  and  especially ^5.5  should 
-be  helpful. 

We  shall  often  use  this  theorem  in  the  following  form: 

Corr^JL:  Under  the  conditions  of  the  theorem  if  has  a  representation 


♦  »  ria9<p>ol 


where  (i)  i  is  the  set  of  masks  of  degrees  £  k,  and  (ii)  a  *  a  ,  if 

9  9 

S(cp)  can  be  transformed  into  S(<p')  by  an  element  of  G. 

Proof:  For  masks,  9A  5  if  *nd  only  if  A  »  Bg  for  some  gcG. 

G 

Coir.  2:  Let  *  ■  - be  the  decomposition  of  i  into  equivalence 

classes  by  the  relation  Then  if  t  is  xii  L(*)  and  *  is  closed  under  G, 

G 

♦  can  be  written  in-  the  form 


♦  "  ^Q^CX)  >  Ol 

where  N^X)  -  1  {9*9*^;  9(X)}\,i.e.,  N^X)  is  the  number  of  (p’s  of  the 
i-th  type,  equivalent  under  the  group,  satisfied  by  the  argument  X. 
Proof:  +  can  be  represented  as 


*  .ftv  >  0] 


mmm 
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t 


t. 


S 


Fe  E.  a{p  cp  >  ol 


i  <pc*t 

•  FE  o  S  «P  >  ol  -  F E  o.M  (X)  >  (jj. 
i  <pc  i  i  i  i 


-,c*  The  Triviality  of  Invariant  Predicates  of  Order  1 

Theorem:  Let  G  be  any  transitive  group  of  permutations  on  R.  (Transitive 
means:  for  every  pairp.q  «  R  there  is  a  gcG  such  that  pg  •  q.)  Then  the 
only  first-order  predicates.,  invariant  under  G  are- 


*<X) 

♦(X) 

t(X> 

♦<X) 


r  lx|  >m1  ^ 
Hx|  *  m "J 
Hx|  <  m  "j 
T(X|  *m\  J 


L  for  some  m. 


Froof:  Since  the  group  is  transitive  all  the  one-point  predicates 
are  equivalent.  Thus  we  can  assume  that 


t.(X)  -  f  E  a  cpr  i  >  e"]  (or  with  some  other 
P«X  inequality  sign), 


i.e. ,  the  coefficient  a  is  independent  of  p.  But  E  a  <pr  i  >  0  can  be 

e  pcX  '■p<’ 

transformed  into  Z*  9^  >-  (for  a  >  0;  for  a  *  0  a  similar  argument 
proves  the  corresponding  assertion).  But  E  9(p}“|x|.  Thus  order- 1 
invariant  predicates  can  do  nothing  more  than  define  a  count  on  the 


• 

o 

1  - 

..•J 


'  \i 


& 


cardinality  of  "area"  of  figures. 


o  ' 


C 

<> 


2.5  Relation  to  Hear 

The  last  theorem  is  closely  related  tn  u.a  v i .  . . 

y  related  to  Haar's  theorem  on  the  unicity 

of  invariant  measures.  ror  measures  «  ... 

'  .or  measures  on  finite  sets  the  unique  ftas* 


measure  is,  ln  fact,  the  counting  function  g(X>  -  |x|.  The  Allowing 

ransrke  make  this  rel.tlon  .  uttl.  more  precise. - 

Ho  first  note  th.t  the  tet  function  defined  by: 


c 


u(X) 


Z*i*i  -  £  a. 

x^tX  1 


is  a  measure,  i.e.,  satisfies: 


M(X)+M(Y)  fi(XUY)  -  h(xOy). 

C 

If  we  defined  invariance  by: 


N» 


C 

sf- 

f 

* 

0 


* 


H(X)  -  n(Xg) 


it  would  follow  immediately  from  Haar 
c  is  a  constant.  Ocr  hypothesis  on  n 
assume: 


'«  theorem  that  p<»  .  c|x|,  uh.ro 
le  slightly  weekor  since  ve  merely 


H<X)  >  o  O— >  p; (Xg)  >  o, 


< 


and  deduce  a  correspondingly  weaker  conclusion)  i.e. , 


(n(X)  >  0)  «->  (c\X  j  >  0) . 

In  the  general  case  the  relation  between  an  invariance  theorem  and  the 
theory  of  Haar  measures  is  less  clear  since  the  set  function  la^^(X)  is  not 
in  general  a  measure.  This  seems  to  suggest  some  generalisation  of  measure 
(perhaps  on  simpllcial  complexes  rather  than  sets)  but  we  have  not  tried  to 
pursue  the  problem.  Readers  insterested  in  the  history  of  ideas  might  find 
it  interesting  to  pursue  the  relation  of  these  results  to  those  of  Pitts 
and  McCulloch  (1947). 

► 

k  - 


t  ■ 


3  , 


CHAPTER  3; 


In  thl.  chapter  ».  study  the  order  of  two  particularly  Intaraattn, 
predicates. 

3-l  Ihe  Parity  Function* 

In  this  section  we  develop  In  some  detail  the  analyal,  of  the 
particular  predicate  defined  by 

’p*S(X)  "  Tlx!  la  an  odd  nuaber] . 

Our  Interest  In  W  1.  threefold:  It  1.  intereeting  in  tta.ll;  It  will 
be  uaed  for  the  analyal.  of  oth.r  wore  taportant  function.;  and, 

especially.  It  Illustrate,  our  m.thm.tlc.1  nethod.  and  the  kind  of  qu.atlon 
they  enable  us  to  discuss. 

Theorem  3.1,1;  *pAR  U  of  order  )&| . 

That  is,  to  compute  tpAR  requires  at  least  one  predicate  whose 
support  covers  the  whole  space  R. 

Proof:  Let  G  be  the  group  of  ail  permutations  of  R.  Clearly  f  i* 
invariant  under  G. 

HOW  auppoae  that  -  ^*1  >  *1  where ^  .re  the  wlth 

|S(9,)  |  s  K  and  the  ^  depend  only  on  the  equivalence  claa.es  defined  b; 

=. 

G 

Since  masks  with  the  same  support  are  identical,  and  sets  with  the 
same  cardinality  can  be  transformed  into  one  another  by  elements  of  G, 


*i“*j  •  lS(Vj)l 


* 


where  4^  is  the  set  of  masks  whose  supports  contain  exactly  j  elements. 
We  now  calculate  for  an  arbitrary  subset  X  or  R, 


C,(X)  -  £  «p(X). 

3  9**3 

Since  <p(X)  is  1  if  Sty)  c  X  and  0  otherwise,  Cj(X)  is  the  nusiber  of 
subsets  of  X  with  j  elements,  i.e. , 

CJ(X)  "{'j1)"  W  <W  ~l>  "  S+1) 


which  is  a  polynomial  of  degree  j  in  jxj. 


It  follows  that 
P(|X|). 


jS0  ajCJ ^  iB  *  P°lynom1*1  of  da8*«e  <k  in|x| ,  say 


Now  consider  any  sequence  of  sets  Xq,  X^,  ....  X|R|  such  that 
contains  i  points,  i.e.,  jxj  ■  i. 

Then  the  sequence  of  values  of  P(|x|), 


PCIXqI)  *<1*^1)  >0*  *<|jy>  *  o . 


F«V>’ 


t 


-3 


changes  direction  |  Retimes  as  |x|  increases  from 
0  to  |r|.  But  since  P  U  «  polynomial  of  degree  K,  it  follows  that 

k  *  |*J. 


q-e.d. 


From  this  we  obtain  the 

Theorem  3.1.2:  If  tpAR  «  L(*)  and  if  *  contains  only  masks,  then  *  contains 
every  mask. 

Proof:  Suppose,  If  possible,  thet  ^  ,  L(i),  tbet  ♦  cooMln.  only  «.ks, 
and  the  mask  whose  support  is  A  does  not  belong  to  *.  Let 


t. 


*  -  f  2  or  q>  >  0]. 

TPAR  q**  <P 


Define,  for  any  *,  +A(X)  -  *(XHA). 

xA 


Clearly  tpAR,  the  parity  function  for  subsets  of  A,  is  of  order  |A\  by 
the  previous  theorem. 


How  consider  cpA  for  <p«#.  if  S(q>)c  A,  clearly  q>A  -  q,. 

If  S((p)  is  not  a  subset  of  A,  cpA  is  identically  zero  since 


S(<p)^.  A  — >  S(<p)£  XH  A  ->  cp(xnA)  -  0  — >  q>A(X)  -  0. 


It  follows  that  either  Sflp*)  is  a  proper  subset  of  A  or  q>A  is  identically 


zero. 


Let  i  be  the  set  of  masks  in  i  whose  supports  are  subsets  of  A. 


r-  tr> 


--<30 


■  r 


a“  *M»  '  ^A  av  »  »  4 


»«  for  .11  ,**»,  |S(  )(  . 

r'>  '*<•  xt  would  follow  that  +u*  . 

*»  l“‘  «■«  I A  I,  mb,..,. _ _  .  .  th‘  °tdM  »f 


♦*  t.  1...  than  I A I  uh<  -  th>t  the  0M“  of 

tAA  th»  |A|,  mhlch  contradict.  theorem  311  _ 

«re  tapoa.uu  .nd  the  theorem  feu  '  '  '  *  h)Wte« 


Corr.  1;  if  *  ,  ....  5*e.d. 

f  W  L<*>  th«n  *  au.t  contaln 

/  S(<P>  |  *  Jr |.  1  one  *  £°r  which 

-r  zzrz  "■  -  — . 

C  iL(i) .  te  ^  ^  *“  ^  f°r  £ESES  “k,«*  *  of  «•  Then 

Ihe  further  analyml.  of  g,  .  _ 

***  be  recognlaable,  1„  pru^l.,  by  7*7  T  'h0VS  *“ 

actiMlly  he  realizable  In  ’  V*ry  perc‘Ptron,  might  not 

bl«  ^  P'octlc.  been.,  of  tapo..lhl.  b. 

F°r  ratio  of  the  l„gMt  „  ” 

coefficient,  of  must  be  2I*I-  1  the  «uMt 

fhe  One-lt>-a.h.'->' 

Another  predicate  of  great  inter...  a 
property  of  "connected,*..."  It,  *  U‘,°cl“M  »ith  the  g.oe»trlc 

“  ^  *  -  «.  theorem  U^ZZ  “ 

1-  •  •  •  •  .  •  ‘‘Joint  .ub..t.  of  »  .„d  define  the  predict. 

HI>  *  rwooxoA^  >on 

*  theorem  is  iu>cw*  riT- 


-  it  f 


*§ 

(  W  tf 


o  ?  ■ 


4  1 

*** 

&  >  I 


'  *  '  . 


!•«•»  there  is  at  least  one  point  of  X  In  each  A±.  Then.  If  for  al 
lA^l.  **  4a  ,  the  order  of  ♦  la  *  m. 

Corr.:  If  R  -  AjU  ...  UA  ,  the  order  of  1  la  at  laaat 

&r. 


i  ° 


A  A,  A,  A* 


The  retina  is  a  square  array 
of  16  cells,  and  A.  is  the  1-th 
coluan.  The  one— in-a-box  predicate 
is  not  satisfied  by  the  figure  to  the 
left,  because  no  cell  in  the  third 
coluan  is  occupied. 


Fig.  3.2 


I 


Proof;  For  each  i  “  1,  m  let  6^  be  the  group  of  penetrations  of  1 
which  permutes  the  elements  of  At  but  do  not  affect  the  elements  of  the 
complement  of  A^. 

Let  G  be  the  group  generated  by  ail  elements  of  the  G^. 

Clearly  t  i«  invariant  with  respect  to  G. 

Let  *  be  the  set  of  masks  of  degree  K  or  less.  To  determine  the 
equivalence  class  of  any  consider  the  ordered  set  of  occupancy  numbers. 


Cls(q>)  fl  Ajj  }. 


» mMMwmjre  «*• 
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i  ii  in  pi 


Mote  that  ^  ^  only  tor  **ch  t* 

Let  *1,  *2»  •••  *u  *•  the  cU,,M* 

Mow  consider  an  arbitrary  aat  X  and  an  equivalence  class  iy  We  wish 
to  calculate  the  m-ber  H^X)  of  -e-bera  of  ^  satisfiad  by  X,  i.e.. 

Mj(X)  -  |  a  S(9)C  X}L 


A  staple  combinatorial  argument  shows  that 


where  Qy  )  -  Y^Y 'pj  ^  «nd  9  is  an  arbitrary  Maker  of  <py 
Since  the  nunber.  1X00^1  depend  only  on  the  classes  «d  add  up 
to  not  more  than  X,  it  follows  that  XjW  can  be  written  as  a  polynoedal 
of  degree  X  or  lesa  in  the  makers  *t  -  JxOAj  : 


l 


J 


r 

o 


Mj(X)  -  •••»  *n>* 

Now  let  ♦  ■  Ts  «P  >  Ol  be  a  representation  of  ♦  as  a  linear  threshold 
function  in  the  set  of  aaeks  of  degree  less  than  or  equal  to  X.  By  the 
argument  which  we  have  already  used  several  ttaes  we  can  assune  that 
depends  only  on  the  equivalence  class  of  9  and  write 


A 


<POO 


‘  A  A,  ■  A  Vj« 

■ 

A  wv  ••••  v 


«hlch.  ...  M  of  polyocml.lt  of  d.p„  «  K>  „  lt..u  ^  . 
polynomial.  Thu.  v«  co  oooclod.  th«  th«.  «i,t.  .  „ 


at  aoit  K, 


polynomial  of  degraa 


Q(a^»  ♦  •  •  *  4t^) 


with  the  property  that 


fq(V  •••»  V  >o  with  x±  «  hnAjl 


i.«. ,  that  if, 


for  ill  i,  o  £  x,  =s  km2 

1  ^  I 


QCxj*  ....  x  )  >o  <—>  (Vi) (x  >o). 

1 


In  Q(xlf  ....  Xji)  make  the  formal  eubetitution, 


xi  "  -  (2i-l))  , 


X 


Th«n  Q(X1#  ....  xj  becomea  a  polynomial  of  dagrae  at  moat  2K  In  t.  Not 


let  t  take  on  the  valuaa  t  »  0,1,  2m. 


Then 


\  *  0  for  «« ■«  1  If  t  la  odd.  In  fact,  for  i  -  -^£-1 
\  >  0  for  all  1  if  t  la  even. 


Hence,  by  the  definition  of  the  *  predicate,  Q  must  be  poeitive  for 
even  t  and  negative  or  aero  for  odd  t.  By  counting  the  number  of  change 
of  sign  it  is  clear  that  2K  >  2m  i.e.,  K  a  m.  Thi.  complete.  the  proof. 


-  m 
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CHAPTER  4:  THE  AND/OR  THEOREM 
Introduction 


In  this  chapter  we  prove  the  And/Or  theorem  stated  in  §  1.5. 

Theorem;  There  exist  predicates,  ^  and  *2  of  order  1  such  that  f  v 
and  ^  /\  +2  *ce  not  °f  Unite  order. 

We  prove  the  assertion  for  t2-  The  other  half  can  be  proved  in 
exactly  the  same  way.  The  results  will  not  be  used  in  the  sequel  and  could 
be  omitted  on  a  first  reading. 

4.1  Lemmas 

We  have  already  remarked  in  f  1.5  that  if  R  -  AUBUC  the  predicate 
*!<*)  -  f  jxnAl,  >  |xac|1  is  of  order  1, 

and  stated  without  proof  that 

ttt)  -  FflxnAl  >  |xncOA(|xnB|  >  |xnc|fl 

is  not  of  bounded  order  as  |r|  becomes  large.  We  shall  now  prove  this 
assertion.  We  can  assume  without  any  loss  of  generality  tlmt 
|a|  «  |b|«  |c|  and  the  formal  statement  is; 

If  1^x)  i8  the  predicate  of  the  stated  form 
for  |r|  *  3M,  then  the  order  of  ^  increases 
without  bound  as  M  -*  ®. 

The  proof  follows  the  pattern  of  proofs  ipf  3.  We  shall  assume  that  the 
order  of  {tM}  is  bounded  by  a  fixed  integer,  N,  for  all  M,  and  derive  a 


-2 


>2 


contradiction  by  showing  that  the  associated  polynomials  would  have  to 
satisfy  inconsistent  conditions.  The  first  step  is  to  set  up  the 
associated  polynomials  for  a  fixed  M.  We  do  this  by  choosing  the  group 
of  permutations  that  leave  the  sets  A,  B,  and  C  fixed  but  allow  arbitrary 
permutations  within  the  sets.  The  equivalence  classes  of  masks  are  then 
characterised  by  three  numbers,  i.e.,  |AAS(cp)|,  jB/">S(<p)  |  and  |crtS(<p)|. 

For  any  given  9  the  nuaber  N^(X)  of  masks  in  its  equivalence  class  satisfied 
by  a  given  set  X  is 


/lv>x|  \  /|bax|  \  / 

Vx>  ■  (  )  x  l  )  M 

'  |Ar»S(cp)  |  /  '  iBnSfv)  I  •  ' 


|BAS(9)  I 


|cr*|  ' 

|crkS(cp)  | 


If  |  F <«p>  |  £  N  this  is  clearly  a  polynomial  of  degree  at  most  N  in  the 


three  numbers 


x  -  |Anx|,  y  -  |b/sx|,  *  -  Icrtfl- 


-v  ; 

v/  » 


Let  *  be  the  set  of  masks  with  | support |  £  N.  Enumerate  the 
equivalence  classes  of  4  and  let  N^(X)  be  the  number  of  masks  of  the  1th 
class  satisfied  by  X.  The  group  inV>  *  Jnct  theorem  allows  tie  to  write: 


♦M(x)  -  p»Aoo  >  oi. 


The  sum  SfijN^X)  is  a  polynomial  of  degree  at  most  N  in  x,y,t.  Call  it 


MsuiBWMSisi 


4 


Now,:  by  definition,  for  those  values  of  x  y  *  - 

<=o  ut  x,y,z  which  are  possible 

occupancy  numbers,  i.e.,  non-negative  integers  ^M: 

PM(x,y,s)  >  o  if  and  only  if  x  >  z  and  y  >  z. 


We  shall  show*  though  a  series  of  I*™., 


that  this  cannon  be  true  for  all 


. 


Lemma  1 


'  j: 


be  an  infinite  aagu.nce  of  „on-«r„  0( 

aegre.  «  vlthtb,  property  t„.£  for  ail  £oaifive_integere  x,y,z 

less  than  M 


and 

x  s  *  2E  V  *  z  ~>  Ps(x,y,z)  £  o. 
am. there  «l,t,  n  ,lnr1.  non.,.ro  „„,vn^„, 

m  5  r  degree  at  m„.- 

iLwltlLthe  property  that.  th«  . 

All  DOAlf*-fv«l  _ «  M 


— — r  .for 

§&  Positive  intPRrai  yalueg  of  M  — 

•  ~  l2U~  Thls  by  a  straightforward 

pnratlon  conditio*,  «,  by  .Uooin,  n,„.llty  i„  both  condltlon|(  ^ 

inequality  would  not  be  preserved  in  the  limit  ron 

lmit’  Consequences  of  this  will 

make  themselves  felt  in 


t  o 

,  0 


wmA 


si 
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rbe  proof  of  lemma  2.)  For  the  sake  of 


elementary  proof.  Write: 


completeness  we  include  the  following 


1’m(x,y>2)  «  £ 


^  St,imi(x*y»‘) 


where  m^,  is  a  enumeration  of  the 


x,y,z. 


monomials  of  degrees  ^  N  in 


-he  conditions  on  PM  are  preserved  under  multiplication  by 
positive  scaling  factor,  we  can  a83Ume  that 


S.i  “  L 


Now  consider  the  set  of  points  in  T- 


•pace; 


"  CcM,1’cM,2,',,,ch,i)*  K  " 


These  all  lie  in  a  compact  set-tha  surface  of  the  unit  T-dim.n.ion.1 
sphere.  There  is,  therefore,  a  subsequence  « 


converges  to  e  limit; 


\  C  (CjjCj, . .. ,c^) 


in  the  sense  that,  for  each  i, 


:  \ 


x 


& 


The  polynomial 


lim  cl^ 

j-»  nj 


CH„i  -  V 


A 

P(x,y,*)  *  £  c.m  (x,y,«) 

1-1  1  1 

Inherit,  the  properties  (A)  for  .11  po.ltive  integr.l  v.lue.  of  »,  y, 

Unit  It  1,  not  identic. lly  eero  folloy,  fro.  the  f.et  thet  the  e,  inherit,. 

the  condition  E  c7  «  1.  1 

i 

Lenina  2 

In  order  to  prove  our  main  theoran,  we  first  establish  a  corresponding 
result  for  polynomials  in  two  variables,  am.  later  (Lama  3)  adapt  it  to 
P(x»y,*) • 

f(a.P)  satisfies  the  following  conditions  for  all 
iPjftgrcU  values  of  g  and  P,  then  it  is  identically  taro: 


^  >  0  ny|  p  >  0  — >  f(a,p)  a  0 

a^Oor  p  so->  f  (a,p)  £  0. 


Proof: 


Assume,  if  possible,  thut  f<a,p)  s.ti.fie.  these  condition,  snd  is  not 
identically  jtero.  We  can  write 


;  i 


f(a,p)  -  p  g(a)  +  r(a,p) 


-?  -  - 


where  r<a,p)  is  of  degree  let*  then  H  in  p,  end  g(a)  i«  not  identically 
*ero.  Then  we  cen  find  e  value  aQ  >  0  such  that  g(a0)  end  g (-c^)  ere  both 
non-zero.  We  can  then  find  pfl  >  0  so  large  that 

0oNg<ao)  >  Ir^.p^J. 

We  heve: 


so  thet 


£<<tyty  >  0  >  f<“V*0> 


g<V  >  0  >  g(-a0). 


It  follows  that  (-p0)Ng(a0)  end  (-ty*g(-ctf  have  opposite  signs.  But 
this  contradicts  the  hypothesis:  P  <  0  —  >  ffa.p)  £  0,  which  implies: 

£<Vpb>  "  <-P0>N  y(o0>  *  0 
f<-V’po>  "  <’Po)N  y(’V  S  °* 

This  contradiction  establishes  the  leans. 

^•2  A  Digression  on  Bexout’s  Theorem 

Readers  familiar  with  elementary  algebraic  geometry  will  observe  thet 
the  lent*  would  follow  immediately  from  Bezout's  theorem  if  the  conditions 
could  be  stated  for  aUjraal  values  of  a  end  p.  We  would  then  merely  heve  to 


O  t 


RQPliir 


prove  that  the  doubly  infinite  L  of  the  figure  is  not  an  algebraic  curve 


Bezout's  theorem  tells  us  that  if  the  intersection  of  an  algebraic 
curve,  L,  with  an  irreducible  algebraic  curve,  Y,  contains  an  infinite 
number  of  points,  it  must  contain  the  whoi*  of  Y.  But  the  L  contains  the 
positive  half  of  the  y-axis.  Straight  lines  are  irreducible,  so  it  would 
have  to  contain  the  entire  y-axis  if  it  were  algebraic. 

Unfortunately,  because  our  conditions  hold  only  on  integer  lattice- 
points,  we  must  allow  for  the  possibility  that  f(a,p)  -  0  takes  a  more 
contorted  form,  for  example  as  in  the  next  figure: 


-8- 


P«rt  of  the  pathological  behavior  of  thi.  curve  i.  irrelevant.  Since 
a  polynomial  of  degree  N  can  cut  a  straight  line  only  H  times,  the  incursions 
into  the  interiors  of  the  quadrants  can  be  confined  to  a  bounded  region. 

This  means  that  the  curve  f(a,p)  -  0  must  "asymptotically  occupy"  the 
parts  of  the  "channel"  illustrated  by 


It  se«s  plausible  that  a  generalisation  of  Besout's  theorem  could  be 
formulated  to  deduce  from  this  that  the  curve  must  enter  the  negative  halves 
in  a  sense  that  would  furnish  an  immediate  and  more  illuminating  proof  of 
our  lemma.  We  have  not  pursued  this  conjecture,  but  believe  it  indicates 
a  valuable  direction  for  future  research. 


u-  -~ 
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“  ‘  P°W*1  ““*«••  *■  «*-*  condition*  fet  ,u 

PO,1MY*  IM,8r*1  V*l““  •*  **•  -  -  «->«  it  i.  Identically  torn. 


x>*«ggy>«  —  >  P(x,y,«)  *  0 

xs*g  yS*—>  P(x,ytM)  z  Q 


Proof; 

SCPPO..  thnt  «*.„.>  had  th...  pcop.tcl...  hot  „t.  not  idant  icily 

'"O'  De£I“  s  P<«  +  «.  .  +  „lt. 


<)(a,;,a)  -  aM  f(a,p)  +  tfo.p.a) 


i«  of  dogreo  1...  than  M  in  a,  and  f(a,»  i.  apt  idantically  taro. 
«  «.  can  .ho.  that  f  «,«  ctlafy  th.  condition,  in  L«u  2;  chca. 

«ny  a_  and  0  for  which  f(a  ,a  \  u  n  r>u 

“0  0  too-V0-  a*,.<  «  .ufflclontly  largo  poaitivt 

*Q»  tor  * 


(a)  *0  *  %  >  0  *0  +  fi0  >  0 

<b)  l*o  ‘<V»o>l  »  lr<V'0-*0l- 


■  »  •  1  ■  ~ 


*,  J  .  *'  C,*  *%*'*&%,*&< 


V 
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U  follows  that  f(a0,»0)  *  0  «->  Q(a0,(l0,,0)  i  0. 
l.«.,  if  snd  only  if  P<t0  +  a,,,  sQ  +  >(),  Ifl)  *  0. 

*"  “0  >  0  ^  f0  >  0  “>  «0  +  <%  >  *0  -*  «o  +  to  >  s0 

“>  *<*o  +  a0’  *0  +  ^o*  *0^  56  0 
->  £<W  *  0 

snd  stallntly  aQ  <  0  2£  p0  <  0  ->  ffcyty  s  0. 

But  this  is  tru.  for  .11  V»0.  Thu.  by  th.  :Lmm.  2.  f(a,W  .  o. 

It  follow.  th«t  P(x,y,n)  i.  of  dsgrs.  s.ro  in  which  is  only  possible 
if  if  i*  identically  zero. 

Thi.  concludes  the  proof  of  the  And/Or  theorw. 


introduction  to  part  it  =  citric  Theory  ^ . 

in  Chapters  5  -  8  we  oil!  seedy  the  proble.  ol  balding  User  predicate 
machines  to  ■•recognise”  patterns  that  are  geo.etric.liy  intereeting.  w. 
mill  study  Chiefly  tUo-di.en.iona!  patterns,  and  will  a.R  ltko; 

(I)  Is  the  proble.  of  deciding  whether  the  input  figure  is  conjas,  or 

is^annected,  of  finite  order  (in  the  sense  of  J  i.6>7  This  is  atudied  in 
Chapter  5. 

«)  »hat  is  the  smallest  order  of  .  perceptron  that  can  recognise 
.Lrianfiles,  or  circles?  Studied  in  Chapter  6. 

<3>  Car.  a  finite-order  perceptron  tell  when  the  input  picture  contain. 

iaa  figures  that  are  congruent  or  sijtiUr  (in  the  Euclidean  sense).  Can 

one  detenaine  which  figures  are  sjn-etric.il  See  Chapter  7. 

(4)  What  can  be  done  with  the  -re  restricted  dlp-ter-li.ir.H 
perceptrons?  See  Chapter  8. 

Our  discussion  of  these  questions  is  not  very  sy.t^tlc 
or  thorough,  because  our  Knowledge  is  still  based  on  too  few  weH-undcr.tood 
particular  cases,  further-re.  we  are  reluctant  to  propose  an,  very  rigid 
classifications  of  the  pledge  we  have  obtained,  because  at  al-.c  every 

S°  ^  teS“US  ^  — d.  Thus,  the  generally  strong 

negative  results  described  in  Chapter  5  lef*- 

“  “nprepar'd  for  the  apparently 

positive  results  in  Chapter  7  which  „ 

’  which  in  any  practical  sense  are  ag.i„ 

reversed  by  the  considerations  in  Chapter  9. 

;or  a  preview  of  the  general  situation,  before  Version  in  mathe-tical 
*  ”e  S“g8eSC  C“dln8  «"*  introductory  section,  of  Chapters  5  -  8. 


Geometrical  Patterns 


We  ere  about  to  study  a  number  of  interesting  geometrical  predicates. 
But  as  a  first  step,  we  have  to  provide  the  underlying  space  H  with  the 
topological  and  metric  properties  necessary  for  defining  geometrical 
figures;  this  was  not  necessary  in  the  case  of  predicates  like  Parity  and 
others  related  to  counting,  for  these  were  not  really  geometric  in 
character . 

The  simplest  procedure  that  is  rigorous  enough  yet  not  too 

2 

mathematically  fussy  seems  to  be  to  divide  the  Euclidean  plane,  E  ,  into 

squares,  as  an  infinite  chess  board.  The  set  R  is  then  taken  as  the  set 

2 

of  squares.  A  figure  ^  of  E  is  then  identified  with  that  set  of  elements 

of  R--i.e. ,  that  collection  of  squares — that  contain  at  least  one  point  of 

2 

Xg.  Thus  to  any  subset  Xg  of  E  corresponds  the  subset  X  of  R  defined  by 

X  *  (xcRjx^Xg  A}. 

Now,  although  X  and  Xg  are  logically  distinct  no  serious  confusion  can 
arise  if  we  identify  them,  and  we  shall  do  so  from  now  on.  Thus  we  refer 
to  certain  subsets  of  R  as  "circles,"  "triangles,"  etc.  meaning  that  they 
can  be  obtained  from  real  circles  and  triangles  by  the  map  Xg  -*  X.  Of 
course,  this  means  that  near  the  "limits  of  resolution"  one  begins  to  obtain 


apparent  errors  of  classification  because  of  tl'e  finite  "i 
Thus  a  snail  circle 


ah"  of  R. 


will  not  look  very  round. 

If  it  were  necessary  to  distingui.h  betw.en  E2  and  R  v.  woM 
that  two  figures  1^,  of  E2  ,r.  lD  R.tol„.„c,  cl„,  Jf 

X  -  X1.  In  thi.  we  would  follow  the  general  nathawatical  approach 

proposed  by  E.C.  Zeem.n  11963)  for  treating  thi.  kind  of  problem; 

far.  we  have  not  had  to  do  ao.  There  i,  no  problem  with  the  tr.nal.tion 

groups  play  the  nain  role,  in  Chapter.  6,  7  and  g.  There  i.  .  .erlou. 

problem  of  handling  the  tolerance,  when  diacuaeing,  a.  iB  |  7.6, 

dilation,  or  rotation..  The  problem  i.  even  more  a.rioe.  when  di.cua.ing 

general  topoligical  equivalence  and  it  i.  only  becu.se  w.  u.e  very  special 

restrictions  on  our  figure.,  i„  ch,pt.r  5,  that  the  tu.rance  theory  can 
be  avoided. 


CHAPTER  5;  CONNECTIVITY,  A  GEOMETRIC  PROPERTY  WITH  mnmimna  n.m»» 

5.0  Introduction 

In  this  chapter  ve  begin  the  study  of  connectedness  A  figure  X  is 
connected  if  it  is  not  co.po.ed  of  two  or  more  separate,  non-touching,  parts. 
While  it  is  interesting  in  itself,  we  chose  to  study  the  connectedness 
property  especially  because  ve  hoped  it  would  shed  light  on  the  more  basic, 
though  ill-defined,  question  of  local  vs.  global  property.  For  connectedness 
is  surely  global.  One  can  never  conclude  that  a  figure  is  connected,  from 
isolated  local  experiments.  To  be  sure,  in  the  case  of  a  figure  like 


one  would  discover,  by  looking  locally  at  the  neighborhood  of  the  isolated 
point  in  the  lower  right  corner,  that  the  figure  is  r»t  connected.  But  one 
could  not  conclude  that  a  figure  is  connected,  from  the  absence  of  any  such 
local  evidence  of  disconnectivity.  if  we  consider  figure,  like  the  following. 


4 


it  is  difficult  to  imagine  any  local  event  that  could  bias  a  decision 

toward  one  conclusion  or  the  other.  Now,  this  is  easy  to  prove, for 

example,  in  the  narrow  framework  of  the  diameter-limited  concept  of  local 

(see  $0.3  and  §8).  It  is  harder  to  establish  for  the  order-limited 

framework.  But  the  diameter-limited  case  gives  us  a  hint:  by  considering 

a  particular  subclass  of  figures  we  might  be  able  to  show  that  the  problem 

is  equivalent  to  that  of  recognizing  a  parity,  or  something  like  it,  and 

this  is  what  we  in  fact  do. 

* 

5-1  The  Connectedness  Theorem 

We  define  connectedness  as  follows: 

Two  points  of  R  are  adjacent  if  they  are  squares  (in  the"  map  X_  -  X 
with  a  common  edge  .  A  figure  is  connected  if,  given  any  two  points 

pl’  P2  of  the  figure,  we  can  find  a  path  through  adjacent  points  from  p^ 
to  p2. 

Theorem:  The  predicate 

KX)  =  Tx  is  connected] 

is  not  of  finite  order  ($1.6),  i.e. ,  it 
has  arbitrarily  large  orders  as  |r|  grows 
in  size. 


*  The  Pr°°f  in$ 5’.1  if  needed  for  the  theorem  of  $5.3.  Otherwise  the  proof 
in  $5.7  of  theorem $5.1  yields  a  better  result. 

t*  can't  ailow  corner  contact,  as  in  ,  to  be  considered  as  connection. 
For  this  would  allow  two  "curved’ to  cross  without  "intersecting": 
and  not  even  the  Jordan  Curve  Theorem  would  be  true.  The  problem 
could  'te  avoided  by  dividing  E2  into  hexagons  instead  of  squares.' 


3 
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<0 


Proof  i:  Suppose  that  <KX)  could  have  order  <  m.  Consider  an  array  of 

,  -  •> 
squares  of  R  arranged  in  2m  +  1  rows  of  4m“  squares  each. 


row  B^ 

m 

m 

mm 

m 

row 

b21 

■ 

■ 

•  •  • 

row  B3 

w, 

fl | 

B 

B 

B 

B 

B 

U 

rou  B2m 

■ 

■ 

■ 

•  •  • 

1 

rOW  S2nri-1 

w, 

Let  Gq  be  the  set  of  points  shaded  in  the  diagram;  that  is,  the  array  of 
points  whose  row  indices  are  odd,  and  let  be  the  remaining  squares  of  the 
array.  Let  ^  be  the  family  of  figures  obtained  from  the  igure  Gq  by 
adding  subsets  of  G^  ,  i.e. ,  F«  if  it  is  of  the  form 
G0vtv  where  F^C.  G^.  Now  F  will  be  connected  if  and  only  if  its 
contains  at  least  one  square  from  each  even  row;  that  is,  if  the  set 
satisfies  the  "one-in-a-box"  condition  of  ^3.2.  The  theorem  then  follows 
from  the  One-in-a-Box  Theorem. 

To  see  the  details  of  how  the  ohe-in-a-bex  theorem  is  applied,  if  it  is 
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not  s Iready  clear,  consider  the  figures  of  fanily  t  0y  ,n 

pottlbm  figures  on  R.  Clearly,  If  „e  had  an  order-k  predicate  *k 
thap  could  recognize  connectivity  on  R,  w.  could  have  one  shat  works  on  t  i 
hweiv  the  sane  predicate  with  constant  eero  Input,  to  all  variable,  not 

ln  V'*Y  And  “U  of  the  odd  row,  have  alway,  value  1  for 

-figures  in  i  ,  thi*  in  turn  mean,  that  we  could  have  an  ordor-k  predicate 
*”  decide  the  one-tn-a-box  property  on  set  ^ nancly,  tho  predicate 
further  restricted  to  having  con, cant  unity  input,  to  the  points  ln  0,, 

Thu,  each  Boolean  function  of  the  original  pradtrtt.  ^  i,  r.pl.cad’hy 
the  function  obtained  by  fixing  certain  of  it.  varlabl.a  to  aero  and  on.; 
thi,  operation  can  never  increa.e  the  order  of  e  function.  But  ,lnoa  thi. 
la.t  predicate  cannot  exl.t,  neither  can  the  original  Iht,  proe( 

show,  that  fa  has  order  at  least  c  •  |*|  .  m  5.7  we  ahow  it  i,  at 

least  c  •  (Rj1/2, 

5.2  An  Example 

Consider  the  special  c...  for  k-2,  and  the  equivalent  one-in-.sbox 
problem  for  a  Gj-space  of  the  form 

In  which  m»3 
*nd  there  are  Juat 
4  square*  in  each 
box. 


k  •f 


Now  consider  a  i|r  of  degree  2;  we  wilL  show  that  it  cannot  characterise 

the  connectedness  of  pictures  of  this  kind.  Suppose  that  t  >  o"l 

and  consider  the  equivalent  form,  symmetrized  under  the  full  group  of 

* 

permutations  Chat  Interchange  the  rows  and  permute  within  rows  .  Then 
there  are  Just  three  cqulvalence-clusses  of  masks  of  degree  :>  2,  namely: 

single  points: 
point-pairs: 
po Int-palrs: 

hence  a»v  >rd«r  2  predicate  must  have  the  form 

+  -  alnl(X)  +  rt11Hll(X)  +  al2Hl2(X)  >  0 

where and  N12  are  the  munbers  of  point  sets  of  the  respective 
types  in  the  figure  X. 

Now  consider  the  two  figures: 


<Pt  - 


•"!}  "  Vj 


(x^  and  Xj  In  same  row) 


12 

<Plj  "  Vj 


(x^  and  Xj  In  different  rows) 


*  Noto  that  this  Is  not  the  same  group  used  in  proving  theorem  $3.2 
There  we  aid  not  use  the  row- Interchange  part  of  the  group. 
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eottHmh 


In  each  case  one  counts: 

Nl  »  6 


,11 


N12  =  9 


»  N  «*  6 

het,=o  the  fora  (1)  has  the  same  value  for  both  figures.  But  X  is 
connected  while  X,  is  not!  Bote  that  here  tn-3  so  that  we  obtaL  a 
contradiction  with  |aJ  -  a,  „hUe  rhe  general  proof  reguired 

|Aji  '  ’  36-  l“  k"°U"  als°  'hat  if  k.6>  „e  pan  get  .  .lBlUt 

esult  with  |aJ  -  16.  This  was  shown  by  Dona  Strauss.) 

Ihe  case  of  k-2,  -3.  Uj  -  3  is  of  order  2.  since  one  car.  in  fa 
express  the  connectivity  predicate  for  that  space  as 

*  -  ftil(X)  +  N12(X)  -  2NU(X)  >  il. 

(This  was  found  by  brute  force) 


hr 
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7. 


* 


The  proof  method  used  in  this  example  is  an  instance  of  use  of  what 
we  call  the  "geometric  ri-tuple  spectrum,"  and  the  general  principle  is 
further  developed  in  Chapter  6. 

5.3  Slice-wise  Connectivity 

It  should  be  observed  that  the  proof  in  $5.1  applies  not  only  to  the 
property  of  connectivity  in  its  classical  sense  but  to  the  stronger 
predicate  defined  by: 

A  figure  X  is  "slice-wise  disconnected"  if  there  is  a  straight  line  L 
such  "hat: 

not  intersect  L  and  does  not  lie  entirely  to  one  side  of  L. 

The  general  connectivity  definition  would  have  "curve"  for  L  instead  of 
"straight  line,"  and  one  would  expect  that  this  would  require  a  higher- 
order  for  its  realization. 

^  fair*y  c^ear  that  human  ability  to  discern  connectivity  is 
limited,  if  the  available  time  is  restricted,  suggesting  a  non-paral’el 
:ess.  Thus  it  takes  a  certain  time  to  decide  which  of  these  figures 
are  connected,  even  in  the  simple  cut-wise  sense: 

0> 

<* 

0 

&  '  <v 


5-'‘  Hsiasuss.  Si  one  Order  Problm  tn 

The  study  „£  the  order  of  pred,M..„  u  oftm  faciutattd  ^ 

given  predicate  to  an.  ..dr  Ampler  one.  Although  we  do 

7  ‘  »  ”£  -  —  -  ~0„.  or  evee  .  ,..t 

enough  lnstght  lnt0  the  Mturc  o£  rel,tlo„s  ^  nighc  ^  ^  ^ 

a-  ogous  ■*—**«.-  "q„tlent„  aad  „  ^ 

^  f0U°“ln8  ~  «  -«  U  ~  .PPUe,lo„. 


O  ! 


> 


and  indicate  an  interesting  area  for  future  research. 

(a)  Let  us  say  that  a  perceptron  syst«,  P,  i,  defined  by  the 
basic  set  R  and  a  sat  ♦  of  predicate,  on  subset,  of  R.  A  ,Mond  p<[rceptc<m 

system,  P',  is.  snb-perceptron  sy.tm,  of  P  If  the  basic  set  R'  t.  .  .ub.set 

of  R  and  if  its  set  of  plicate.  V  is  that  obtained  by  rel.tiviaing  the 
members  of  *  to  R' ,  i.e.,  each  predicate  satisfies 


XCR’  —  >9'(X)  -  <p(X)  for  some  epe* 


and  all  predicates  cp'  satisfying  this  condition  are  in 

Clearly  the  order  of  any  predicate  of  the  form  f  for  P'  is  at  most 
that  of  ijr  for  P. 


(b)  Isomorphism  must  be  given  the  folloving  natural  sense:  let  ¥ 

be  defined  by  R  and  i  and  P'  by  R»  and  $•  Th»r.  ...  , 

y  ana  *  .  Then  an  isomorphism,  f,  is  an 

isomorphic  map  f:  R  -  r*  0f  the  aet..  „ 

or  cne  setjs  R  >?ith  the  property  that  for  each 

ft*  there  is  exactly  one  <p'e*  satisfying  <p(X)  -  <p'<f(X»  („hsr. 

f(X)  »  {p«R'|  q6R;  f(q)  »  pjj 

(c)  R'  is  obtained  from  R  bv  a  ml io„.i„„ 

Dy  a  coUapaing  operation  f,  if  f  u  a  Mp 

from  points  of  R'  to  disjoint  sets  of  R,  i.e. 


P6R'  “ >  f  (p)  C  R 


p*q  ->f(p)r|f(q)  -  A. 
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A  predicate  tjr*  on  R'  is  obtained  from  a  predicate  f  on  R  by  the 
collapsing  nap  f  if  *'(X)  »  t(f(X').  For  X'CR'. 

Theorem  5-4. l:Collapsing  Thaoran; 

If  f  is  a  collapsing  map  from  R  to  R*  and  is  obtained  from  *  by  f, 
then  the  order  of  ;  greater  than  that  of  *. 

Proof:  Let  if  -  .  where  $  is  the  set  of  masks  of  degree  less 

than  k  on  r. 

Now  for  any  X'  C  R, 

f(X')  -  t(f(X’» 

-  fEa^CW))  >01. 

We  observe  that  (1)  remains  true  if  f  is  replaced  by  the  set  $ 
of  masks  cp  for  which  S(cp)  Cf(R'),  for  if 


S(<p)  ^f(R')  then  cp(f(X'» 


0  for  all  X'  C.  R. 


Now  for  cpe$  we  have 


S(cp)c  L/  Cf(p)  |PeR' }, 

in  fact 


S(9)C  U  Cf(p)  Jf(p)nS(«p)  ¥  A}. 


«SfV**C* 


'I 


% 


Thus, 


■n 


W 
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*’  Cp|Cp)^sc<p>  *  A} 

->  U*')Z>  U  Cf(p)|f(p)nS((p)  t  a}ds(<p). 


i.e.,  X*3  {p|f(p)nS(«p)  *  A  —  >  ((I')3S(f)  — >  q>(f(X')). 

On  the  other  hand,  if  <j>(f(X')),  i.e.,  f(X'DS(?),  it  follow,  that 


f  (p)  n  S(<p>  i  a  —  >  pcx' 


c 


since  f  (p)  ^  f  (q)  -  A  for  p  f  q. 

Thus  <p(f(X'))  -  fx*  O  (p|f(p)  A  S(<p)  f  A}*|. 


In  other  words  cp(f(X'))  is  a  mask  on  R'  with  support 


Cp|f(p)n  S(«P)  t  A}. 


\ 

\ 

I 

1 


But  since  the  sets  of  the  form  f(p)  are  disjoint,  for  different  p,  it 
follows  that 


|Cp|f(p)r\S(sp)  i  A}  |  s:  js(<p)|  <k. 


Going  back  to  equation  (1)  we  see,  then,  that  is  represented  as  a  linear 
function  of  masks  of  degree  less  than  k. 

q.e.d. 


J. 
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=•5  Huf^n-s  Con.tr.njM.n^ot^  p,„„f  2  of  ,  , 

**  ‘h"U  IUU,C”te  the  «P^*«on  of  the  proofing  cooc.pt  by 
giving  an  alternative  proof  that  ha.  no  fintt.  order,  baa*  on  a 
construction  suggested  to  us  by  D.  Huffmin. 

The  intuitive  idea  la  to  conatruct  a  .witching  network  which  will 
he  connected  if  an  even  nunber  of  it.  n  awitche.  are  in  the  ...... 

poaition.  Thu.  the  connectednea.  proble*  i.  reduced  to  the  parity 
problem.  The  network  is  shown  in  the  diagram  for  n  «  3. 


1/ 


The  interpretation  of  the  ay.be  1  a  ^  a™,  ^  i.  follous;  u  ^ 

«.e  .W.  poaition  contact  i.  nwd.  whenever  x,  appear.,  and  broken  whenever 

Xi  aPP“rSi  “he"  *1  ls  in  the  "»«"  Position  contact  ia  nade  where  J 
appear,  and  broken  where  xt  appear..  It  i.  eaay  to  aee  that  the  who!,  net 
ia  connected  in  the  electrical  and  topological  acne,  if  the  nunber  of 
switchee  in  the  "on"  poaition  i.  0  or  2.  The  gener.lit.tion  to  n  i.  obvioua- 
(a)  Liat  the  tern,  in  the  cl.aaic.1  for*  f„r  ^  conaidered 


>  " 

ir 
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as  a  point  function,  which  in  the  present  case  can  be  written: 


PAR  (x1>x2,x3)  «  X1X2X3 v XiX2x3 v x1^2X3V/^lx2x3 


(b)  Translate  this  boolean  expression  into  a  switching  net  by 
interpreting  conjunction  as  series  coupling  and  disjunction  as  parallel 
coupling. 

(c)  Construct  a  perc^ptron  which  “looks  at“  the  position  of  the 
switches. 

The  reduction  arguement,  in  intuitive  form,  is  as  follows:  the 
Huffman  switching  net  can  be  regarded  as  defining  a  class  &  of  geometric 
figures  which  are  connected  or  not  depending  on  the  parity  of  a  certain 
set,  the  set  of  switches  that  are  in  “on"  position.  We  thus  see  how  a 
perceptron  for  on  one  set,  R,  can  be  used  as  a  percept ron  for  \^T1All  on 


TPAR 


a  second  set  R' .  As  a  percept-ron  for  tpAR>  it  must  be  of  order  at  least 


|R'|.  Thus  the  order  of  mast  be  of  order  |r*  (.  We  can  use 


the 


collapsing  theorem  to  formalize  this  argument.  But  before  doing  so  we 
note  that  a  certain  price  will  be  paid  for  its  intuitive  simplicity:  the 
set  R  is  much  bigger  than  the  set  R' ,  in  fact  |r|  must  be  of  the  order  of 

|R’I 


magnitude  of  2  so  that  the  best  result  to  be  obtained  from  the 

construction  is  that  the  order  of  must  increase  as  log  JrJ. 

This  gives  a  weaker  lower  bound,  log  |r)  compared  with  |rJ if  we 
wish  to  estimate  the  order.  J.n  fact,  in  order  to  escape  this  penalty  we 


1 

V 

•  ■  m 


I  v 


u 
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decided  not  to  apply  the  collapsing  theoret  after  ell  to  th. 

Instead  we  shall  construct  a  related  but  ^  case: 

obtain  a  sharper  bound.  **  COaple*  swltchin8  net  to 

5‘6  ggnnectivity  on  a  Toroidal  Spa™  |d| 

Our  earliest  attempts  to  prove  that  *  h 

Connected  as  unfcounded  order  led 
to  the  following  curious  result;  The  predicate  i  n 

^CON  on  an  2n  x  6  toroidal! v 
connected  space  |r  l,as  „rdar  e  Ihe  " - 1 

pr°o£  ts  b*  c?astruction:  consider 

the  space 


Figure  5.6.1 

in  which  the  edges  ...  and  *.f  are  identified.  Consider  the  family  of 
subsets  of  R  that  satisfy  the  conditions 

(i)  all  the  shaded  points  belong  to  each  X«^ 

(ii)  for  each  Xe^and  each  i,  either  < 

’  eicner  both  points  marked  a 

or  both  points  b.  are  in  &  h„*  * 

i  are  m  cT,  but  no  other  combinations 

nre  allowed. 

Then  it  can  he  seen,  for  each  X*  that  X  has  either  one  connected 
Opponent  or  X  divides  into  two  separate  connected  figures,  which  cate 
actually  occurs  depends  only  on  the  parity  of  (£*,  ,x)|.  Then  „.i„s  th. 

oollasping  Theorsn  and  Theor«/,,l,  we  find  that  ^  has  order  ^ 
The  idea  for  this  proof  c«  fron  the  attempt  to  reduce  connect^,.. 
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to  £arit£  directly  by  representing  the  switching  diagram: 


Figure  5.6.2 

If  an  even  number  of  switches  are  in  the  "down"  position  then  x  is 
connected  to  x*  and  y  to  y\  If  the  number  of  down  switches  is  odd,  x  is 
connected  to  y'  and  x*  to  y.  This  diagram  can  be  drawn  in  the  plane  (see 
$5.7)  by  bringing  the  vertical  connections  around  the  end;  then  one  finds 
that  the  predicate  Tx  is  connected  to  x'T  has  for  order  some  constant 
multiple  of  |r|  .  If  we  put  the  toroidal  topology  on  R,  the  order  can 

be  shown  to  be  greater  than  a  constant  multiple  of  |R|;  this  is  also  true 
for  a  3-dimensional  Euclidean  R.  These  facts  strongly  suggest  that  our 
bound  for  the  order  if  *C£)N  is  too  low  in  the  2-dimensional  plane  case.  The 

following  section  improves  the  situation  somewhat  by  replacing  |R|1/3  by 

|R|1/2. 

5.7  Reduction  of  in  the  Plane 

The  following  construction  shows  that  the  order  of  f  is  z  0(|r|1/2) 
for  two-dimensional  plane  figures.  It  results  from  modifying  Figure  5.6.2 
so  as  to  connect  *  to  x’.  This  is  easy  for  the  torus,  but  for  a  long  time 
we  thought  it  was  impossible  in  the  plane. 


1 


to  be  a  pair  of  figures  with  the 


We  first  define  a  "A- level  switch" 
following  two  connection  diagrams. 


Figure  5.7.1 

In  the  "down"  state  we  have 

pQ  connected  to  q^^ 

Pj.  connected  to  q2 
P2  connected  to  q3 
P3  connected  to  qQ 

and  we  write 


Them  changing  a  switch  will  have  the  effect  of  adding  2 (mod  4)  to  the 
index  of  the  q  connected  to  any  given  p,  because  subtracting  2  has  the  same 
effect  as  adding  2.  Now  consider  the  effect  of  n  switches  in  cascade: 


This  simply  iterates  the  effect:  each  switch  that  is  "down"  adds  1  to 
the  q- index  and  each  "up"  switch  subtracts  l(mod  4)  so  that  if  k  switches 
are  down  we  have 


^i  >  ^i+k-(n-k)mod  4 

Then  that  there  are  only  two  possible  mappings  since 
if  an  even  number  of  switches  are  down  we  have 


Pi  >  qi-n(mod  4) 


and  if  an  odd  number  are  down  we  have 
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Pi  qi+2-n(mod  4)‘ 

Finally,  we  add  fixed  connections  tying  together  p^,  p2  and  p3  and 
^l+n’  q2+n’  and  %fn’  The  overall  effect  in  the  two  cases  is  then  either 


Figure  5.7.3 

and  we  see  that  in  the  one  case  the  network  is  disconnected  and  in  the  other 
case  it  is  connected.  Me  illustrate  the  n  -  6  case: 

(See  Figure  5.7.4,  p.  16c.) 


and  we  can  stats: 

Theorem  / . 1:  This  network  is  connected  when  an  odd  number  of  switches 
are  down  and  disconnected  when  an  even  number  are  down. 

It  remains  only  to  realize  the  construction  of  the  switches.  Define 
a  switch  to  be  the  two.  configurations: 


•  1 
j  r 


Figure  5.7.5 

Bemember  that  is  not  s  connection.  When  the  entire  construction 

is  completed,  for  n  switches,  the  network  trill  be  shout  5n  squares  long 

and  about  2n  +  12  squares  high,  so  that  the  number  of  switches  can  grow 

.  l1/2 

proportionally  to  .  |R|  It  follows  that  the  order  of  grows  at 

1/2 

least  as  fast  as  |r| 
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Thc  idea  for  the  proof  comes  from  observing  that  in  the  planar 
version  of  Figure  5.6.2 


other.  If  we  could  make  a  permanent  additional  direct  connection  from 
P1  to  ql  then  the  whole  net  would  be  connected  or  disconnected  according 
to  the  parity.  But  this  is  topologically  impossible,  and  because  the 


construction  appeared  incompleteable  we  took  the  long  route  through  proving 
and  applying  the  One-in-a-box  theorem.  Only  later  did  we  realise  that  the 
P1  **  ql  connection  could  be  made  "dynamically,''  if  not  directly,  by  the 
construction  in  Figure  5.7.1.  This  figure  is  made  by  superimposing  two 
copies  of  Figure  5?7.2,  and  using  the  second  copy  only  to  insure  that  ^ 
and  q2  are  always  connected  in  the  first  copy. 

C 


What  is  the  true  order?  Let  us  recall  that  at  the  root  of  the  proof 
methods  we  used,  was  the  device  ($5.0)  of  considering  not  all  the  figures 
but  only  special  subclasses  with  special  combinatorial  features.  11ms  even 
the  order  |r|/12  of  $5.6  is  only  a  lower  bound.  Our  suspicion  is  that  the 
order  cannot  be  less  than  |r|/2.  As  for  the  number  of  cp* s  required , 

Theorem  3.1.2  and  the  toroidal  results  give  us  Js  2^^,  but  his  too,  is 
a  low(  bound,  and  one  suspects  that  nearly  all  the  masks  are  needed. 

Another  line  of  thought  suggests  that  one  could  get  by  with  the  order  of 
the  number  of  connected  figures,  but  that  has  probably  not  much  smaller 
exponent.  As  for  the  coefficients,  the  results  of  $9  will  apply  ismiediately. 

Examination  of  the  toroidal  construction  in  $5.6  might  make  one 
suspect  that  the  result,  ^  a  ^  JrJ  is  an  artifact  resulting  from  the 
use  of  a  long,  thin  torus.  Indeed,  for  a  "square"  torus  we  could  not  get 
this  result  because  of  the  area  that  would  be  covered  by  the  connecting 
bridge  lines.  This  clouds  the  conclusion  a  little.  On  the  other  hand, 
if  we  consider  a  three  dimensional  R,  then  there,  is  absolutely  no  difficulty 
either  in  the  torus  or  in  ordinary  E3  of  showing  that  +  «s  -  IrI. 

leave  unresolved  the  problem  of  finding  precisely  the  lower  bound  of 

2.  00,1 

in  E  ,  content  with  showing  that  it  is  not  of  finite  order,  and  that  it 

1/2 

grows  at  least  as  fast  as- |r|  -  . 

5.8  Predicates  Related  to  the  Euler  Formula 
Curiously  enough  the  predicate 


/•** 


Tx  is  connected^  contain.  et  ,»«  one  ^ 

has  finite  order,  even  though  neither  di.ilffct  doe.-en  in.tenee  of  tBe 
opposite  of  the  And/or  Phencenon.  Thi.  win  be  .hom  by  . 
voiving  the  Euier  reunion  for  orient.ble  glottic  £1gures. 

5*8*1  Euler  Polygon  For»„l, 

Too-dimen.ion.i  object,  heve  .  topoiogitei  inp.ri.nt  G(x)  ^  ^ 
polygonal  cases  is  given  by 

C(X)  -  |p.ce.(X),|  .  |gdge.(g)  j  +  j Vertices (X)  | , 

Some  examples: 


I 
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Topologically,  0(1)  1.  ln  glvra  by 


c(*)  “  {connected 


component* |  -  (hoi  es|. 


It  is  possible  to  ask*  lownr>l>* 

r«n  *“  c“lI“  *"«— 

like  ( G(x)  .1  -nd  &«)  <  follo|f# 

For  each  point  x£  of  &  chooae  weight 

ai  "  +1  ("vertices"). 

For  each  adjacent  pair  j| 

aij  "  _1  ("edge.*") . 

g  Ch«,., 

°ijkl  "  +1  ("feces") 

For  all  other  .a*.,  chooae  weight 

a  «  0. 

Then  it  is  claimed  that 


«*ooae  weight 


;  ~  «*k> - 


ri*j5k*r 


Wa  .ketch  the  proof  by  (Inductive)  analyal,  of  «...  h. 

«-  *  A  i.  0*.  to  a  figure.  ^ 

ao  -Jaceot  a, oar.  to  a  figure.  that  touch.,  o.  ou,y 
*“  -*  Ch“"  <=•  -  **.  1  -  1  -  0  to  our 


mC.  u. 


•-guano..  .  a^j 
»  . 
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Adding  a  square  that  touches  two  others  normally  decreases  G  by  unity, 
and  appropriately  adds  1  -  2  «  -1  to  our  stm: 


u 

This  normally  connects  two  components  or  (if  the  two  were  already  connected) 
adds  a  hole.  The  exception  is: 


which  does  not  change  G  and  appropriately  adds  1  -  2  +  1  »  0  to  the  stan. 

Case-analyses  of  the  3-neighbor  and  4-neighbor  situation  complete  the 
proof:  these  include  partial  fills  like 


which  add  1  -  3  +  2  -  0  and  1  -  4  +  4  -  1,  the  latter  representing  the 
increase  in  G  when  a  hole  is  finally  filled-in.  All  this  corresponds  to 
an  argument  in  algebraic  topology  concerning  addition  of  edges  and  cells 


j 
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to  chain-complexes. 


It  follows  immediately  that  the  predicate 

fc-(X)  <  nl 

is  realized  with  order  S4  j 

leads  to  some  curious  observations-  If 

we  are  ^iven  that  the  fibres  *  ^ 

figures  X  are  restricted  to  from 

.  co  trom  the  connected 

(-  «*».  order.4  MchlM  can  tecogni2e 

is  ®*®ply-connect«d*|  -Ig(X)  >  ol 

or 

F*  has  less  than  3  holes-}  .  (e(J)  >  .  2-j 

Bet  Of  course  we  oa„„„t  concede  that  these  can  be  - 
by  a  finite  order  perceptron.  *" 

-at  this  topological  invariant  is  the.  seen  to  he  highly  .w. 
in  nature— indeed  all  the  m's  ,  *  1 

y  <a  Very  “Sht)  diameter-limitation' 

OW  returning  to  our  initial  claim  we  note  that 

rG(X)  =  nl  a  (rG(X)  a  nl  a  r®«)  s  jjJ 

h«ce  by  Theoran  1.5.*  we  can  conclude  that  ROO-SI  has  order  .  8.  But 
'  ^  *  ““  constructing  product-,,  that  are 


V  V 
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diameter-  limited  ,  and  we  believe  that  rM 

diameter-limited  ‘M*  Predicate  cannot  be  reelieed  b, 

tied  perceptrons.  Ih.  situation  , 

between  <8.2  and*,. 9,  buc  h,ve  M 

true,  we  have  the  i„t  *  If  ****  COnJecture 

tbe  intriguing  Possibility  o,  u.ing  the  theory  to  cl.s.if 

topological  invariants  into  different  categorie.  of  , 
be  the  analogy  if  a„  .  c  localnese."  What  would 

-logy,  if  any,  with*,  top„iogy,  of  the  di.tl„cti„„  b„r 
diameter-  and  order-limitedf 

5'8'2  SBiffieneg,  of  Tonn1„.,-,i  . . 

—  ““  “  — — 


I 

i 


L({x 


i* 


Vj* 


X^x 


jVi^) 


1<V 


0  contains  the  masks  enumerated  in  5  8  l  u 
assuming  bounded  coefficients  (see ^9.  ).  ^  ^  ^ 

aa£SLi^:  logical  invar . f 

- -===■ *  in  Lf*n)ye  function, 


6(X). 

^£2£f:  Consider  any  figure  x 


right.  Then  the  sequence  of  figures 


«d  let  p  be  one  of  its  poin£8 


»«ti«aliy  to  the 


s 

i 


<1 
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% 


K 

i 

> 

I 


•re  topologically  equivalent. 

Suppose  that  ieL(*  )  i*  , 

<  0)  i«  Mpolo.tc.Uy  lnv.rt.nt,  stop.  thl.  , 
tr.n.l.tlon  «nd  rot.cloo  (by  90°)  ,  in5lud“ 

W  90  >  i"*«Mooco,  »«  c,„  „u, 

♦ .  r^D  +  S2Ptn  +  Yaf  >  n 

D'flne  f(D)  *  *  th'  *“  ‘H.  «~.«oo.  for  fi^iro  X  Si 

0£  £he  lnc,u.Uty  ,utt  be  £he  w  £m  »•  i0“  th* 

ell  n.  1  Xn  we  Bust  have,  for 


f00  +  n(a  +  p)  >  8 
and 

f(Y)  +  n(a  +  p)  <  e 


for  s"y  fl8urM  *  - y  t(x, _  ky).  lut  th 

cc  +  6  «  o  k  “hen  we  must  have 

’  C8USe  0therWl8e  *  would  b«  trivial  (constant). 

ily'  by  *PPMdiag  to  X  the  fljtutt. 


one  finds  that  2a+3p  +  v«fi4.„ 

P  +  Y  -  P  +  (Y  -  o  and  we  conclude  that 


§ 


and  hence  that 


*  -  Tg(X)  > 


Hence  L(iQ)  contains  only  the  Euler  invariants' 


Q.E.D. 

He  can  push  the  argument  into  larger  Vs.  Suppose  that  the  masks  of 
the  formr-Jiare  adjoined,  to  4^.  *£hea  by  usins  the  figiu  -extensions 


-SB 


#  A 


we  find  that  2a  +  2p  +  6  ■  6  »  0,  hence  the  new  masks  will  have  rero 
coefficient  in  j.  Sven  if  we  further  adjoin  the  matkcP  »  we  must  then 
have-  (from  the  same  extensions) 


“a  +  “CD  +  “Cfi  +  “cP 


0  *  a  n  +  a 


a  +acP 
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and  cMPlex 


sequence  of,  extensions. 


comparing 


shows  tttU  A  .  _ 

*  *  ci jp  %  an 

«-•»  4.  ,•  „  "** 


-  Hence 

tMt;  CiCrj  SbJ  henr*  ~ 

hence  o r  j  are  both  aero  (*,. 

a.  -  a.  « « 


Would 


r.  «e  both  zero  (el,e  *■ 

1  ^  SU»6't*  *  «»J«cta„  w,  h.ve  f  '  1 

*****  of  flnit  “  •»* «-« 

Eulerl“  %  «.  ptove<i,  ~  ,,  '  °rdtt  *"  “f 

theorra'  CW  *t  Wouldn't  give  a  iwply  th.  connected,,,., 

“■gnitude  estimate  on  otd„.,rovth. 


£% •»;  :Tr  of 

oie  within  another 


1 


1 


4 


XI 


\ 
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Chapcei‘  6  *«■  ?  -in  d«„„strate  some  techniques  .  , 

ecnniques  for  ciji.tructing  low- 

predicates  of  geometric,  character.  The  result. -.r.  0 

«-  —  of  the  previous  two  chapters  „  "" 

to  find  thst  certain  vredic  t 

prediCateS  “*  °f  —  ^wer  order  then  we  „ri8,„.Uy 

xpected.  However,  there  are  also  some  negative  result,  „f  a  n„  klnd 

^  “  *h‘U  £“'  *  — — fh,  patt.ru.  cdnt;t7;  ID 
low-order  perceptrou  can  tell,  i„r  example  whether  a  given  ilgur,  t.  . 

^  ^  "7-2-5)  b“C  tha  «f  deciding  whether  a  flg„re 

^£4iM  o  oeeare  (and  p,rhap.  something  else,  is  m  of  £lnito  ^ 

In  pter  1  we  describe  a  very  powerful  technique,  "str.tiflc.ti  „ 
that  shows  how  to  construct  finite-order  fun  ,  £ 

under  important  geometric  groups  e  g  t  ““ 

th  ,  ‘8"  translat*°”  and  dilatation..  But  thi. 

wothod  seem,  to  give  rise  to  extremely-  large  coefficient..  In  Ch.ptct  , 

ooe  that  the  occurrence  of  coefficient,  which  increase  without  hound, 

'  ^  rC“”  lnCr“‘C“’  “  ~  “  -“-“1  hy-product  of  this  method 

7  ^  “  “1U  *  **“  ““  predicates  as  simple  a.  recognising  a 

single  given  figure  independently  of  translation  on  the  retina  „ec,...rfly 

::::r  in  -  ~ 


The  division  of  material  between  Chapters  6  and  7  correspond,  fonsally 
general  method,  we  have  found  for  constructing  geometric  predicate, 
of  -finite  order,  "dlfference-vector  spectre"  (chapter  6"  and  "Stratification" 
(Chapter  7,.  A  deeper  difference  is  related  to  the  .fa  . .  of  gto„p  ^ 


‘4 


m 


2- 


C 


in  the  two  esses. 

The  geometric  predict.,  we  went  to  confer  ere  a> 

sre  d'  nvariant  under 

Che  group  of, .Euclidean  transformation.,  and  maietim 

size-dilatation  a.  well  th  8  "*« 

well.  the  groups  of  permutation  used  up  to  now 

dilatation  1.  not  easy  to  define  in  the  content  of  flntte 

The  difficulties  are,  .t  lea.t  . 

uperficlally,  „f  two  kinds:  those  that 

nr  - — -  ••  — -  - 

» ...  - r.~  “  r  -  — -  ~ 

gure  on  a  discrete  retina,  but  ve  mav 
be  prevented  by  the  of  *  y 

y  cne  raster  size  from  halving  it 

kin.  _  ^  8  Worse  Problems  of  the  sMe 

cur  when  we  consider  rotation,  (other  than  those  thet  preserve  the 

iength  arisen  connection  with  the  .re  innocuous  group  of 

.  ,  1P“e  tHC  det*11'  »£  and  7  by  a  brief  tra.ry  of  the 

-in  results,  m  chepter  6  we  use  ft  that  in  . 

Che  "local"  structure  of  the  group  yhe  ^  “ 

„  U  The  theorem.  we  prove  then  ra»al„  £r„e 

whether  R  is  the  full  infinite  pl.„e,  or  if  we  force  it  to  be  fi„it.  ,  , 

a»  »e  did  in  fe.6  by  cutting-out  .  finite  portion  a„d  ,ew!„g  (t.  ,dgei  .  ”  . 
with  a  fnrnij. i  g  i  *  edge t  together 

^  connection.  i„  all  cases,  the  Group  Invariance  theorem  i. 

applicable,  and  the  coefficient,  ere  e,„sl  0„  eguivalent  etc  The  Bri 

-  "  "  ^  ~  ~  -  —  *  -eter-limitation  ir 

n  the  predicate,  or  in  the  cl...  0f  figure,  to  be  accepted.  (Ih„.  w, 

31;; the  -  - — * ... 

cal  diameter- limited  property  of  having  only  four  corners.  ,„t  „  c.ntmt 
ognize  geometric  s,„.re.  (presn^bly,  without  the  method,  of  chapter  7  , 


In  Chapter  7  we  uee  more  global  properties  of  the  transformation 
group;  in  particular,  that  ail  the  translations  of  the  plane  can  be  orde^J 
in  an  enumeration  under  which  sufficiently  large  elements  always  dominate 
sufficiently  mall  ones.  These  enumerations  enable  us  to  show  that  some  less 
local  properties  can  be  realized  on  the  full  infinite  plane:  the  price  is 
unbounded  coefficients  and  loss  of  equal  coefficients  for  equivalent  predicates. 
These  procedures  do  not  survive  toroidal  closure  of  a  finite  portion  of  the 
Plane  because  the  required  property  of  the  ordering  is  destroyed,  and  this 

appears  to  be  irreparable  if  the  group  contains  cyclic  group,  in  it.  global 
structure. 

The  predicates  and  methods  of  representation  of  Chapter  6  depend  only  on 
the  local  structure  of  the  group.  In  the  next  chapter  we  use  more  global 
properties  of  the  transformation  group;  for  example,  we  will  use  a:  complete 
ordering  of  all  translations  to  obtain  low-order  representations  for  certain 
predicates.  This  result  doesn't  carry  over  to  rotation  because  these 
procedures  do  not  survive  a  toroidal  closure  of  a  finite  part  of  the  plane. 

(The  ordering  relation  is  destroyed.*)  This  seems  irreparable  because  the 
group  now  contain,  a  cyclic  subgroup  in  its  global  structure.  Indeed  we  find 
(in  97.10)  that  certain  simple  translation-invariant  predicate,  do;  not  satisfy 
the  group  invariance  theorem  on  the  infinite  plane.  It  will  turn,  out  that  we 
can  use  this  to  advantage.  Ve  eventually  shall  show  (in  §9.4)  that  the  C.I. 
theorem  does&hold  if  bound,  can  be  placed  on  the  coefficients,  and  so  deduce 
that  the  delinquent  predicates  cannot  possible  be  represented  with  bounded 
coefficients.  But  despite  this  consolation  prize  we  feel  deeply  dissatisfied 


of  °f  the — -  -  . 

«  u  eon.  deeper  fact  connected  with  the  giobal  structure 

‘  8t0“P>  tUt  “  -  -U  -  goer,  precisely  whet  it  is.  nu. 

although  the  next  two  chapters  contain  * 

intri  ,  _  -V  causing  construction,  end 

intriguing  theorems,  we  see  - 

*  the.  a.  posing  TOre  question.  rhan  they  answer 

Because  translation-invaridnce  is  required  thr  w 

required  throughout  Chapter  6, 

.  i  ,  - -  in  fcac*1  equivalence-class  This 

is  always  (trivially)  true  for  finite  8  «„r  «, 

preceptron  counter-cpU,  t  ”  th'"  “1*f  l0finitc 

unter  copies  to  ease  „f  the  statesent.  in  chapter  , 

"  “•  * — -v,  we 

fors,.  even  after  ^  **  ‘  *"  th*lr  »“« 

covering  the  methods  of  Chapter  7 

4-u  .  ““•peer  /,  because  in  real  lif. 

iHe  conclusion.  Based  „„  th.  finite  case  are  the  „.t  Warrant  taev  ’ 

interesting  the  infinite  case  night  he.  nath.atic.lly. 

infc.l-fc.d.eshow  that  certain  patterns  have  orders  -  i,  .  2  SJ 
-  tespectivel,  X„  no.t  case,  we  usual*  hay.  «  established  the^ 
end  on  the  order,  and  have  no  sv.tn.stio  nethod.  for  doing  ~~ 

1  ^5£tri£  Patterns  of  nr^  i 

When  we  say  "geometric  property"  we 

P  operty  we  mean  something  that  is  at  least 

invariant  under  translation,  usually  also  inv-  < 

.  y  invariant  under  rotation,  and  often 

,7  —  -  —  -  — —  conbine  to  define  t  e 

th7  8ro“P  °£  tr“*£~  ~  —  .lib.  the  figure. 

£D  EUCl““n  »«  —  1  we  ^w  that  .11 


-  # 


coefficients  can  be  assumed  to  be  equal,  since  the  translation  group  satisfies 

the  condition  for  Xheorw,  2.4.  Therefore,  the  only  pettern.  that  ten  be  of 

order  1  are  those  defined  by  a  single  cut  in  the  cardinality  or  are.  of  the 
set: 

*"  W  >Al  or  t-  Tjxl  <Al. 

**°te*  ^  translation  invariance  is  not  reauirerf 

noc  required,  then  perceptrons  cf  order  1 

can  compute  other  properties,  e.g. ,  concerning  moment,  .bout 
points  or  ares.  However,  these  are  not  "geometric"  in  the  sense  of  being 
sultablly  invariant  so  while  they  way  be  of  considerable  practical 
importance,  we  will  not  discuss  than  here*. 

6,2  Patterns  of  Order  2.  Distance 

For  k  =  2  things  are  wore  complicated.  A.  shown  ln  §1.4,  „  m  u 
possible  to  define  a  double  cut,  or  segwent,  in  the  are.  „f  the  aet;  that  ia, 
we  can  do  the  counting  trick,  and  recognise  the  figure,  whose  area,  are 

+  “  rAi  <  Jx|  <  A^. 

In  fact,  in  general  we  can  always  find  a  function  of  order  2  k  that  recognite. 
the  set.  whose  areas  lie  in  any  of  k  intervals.,  B„t  let  us  return  to  pattern. 

6'g"  HCCUU°Ch  ^  Plt“  <1M7>  “  ey*-centering  servowechanisw. 


* 

^  * 


r 

\ 
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iant  under  geoaetric  group*.  First,  consider  only  the  group  of 

aPi  “**  °£  2-  «.  xtx2  and  . 

equivalent  if  and  only  if  the  difference  vectors 

xi  “  xo  and  x, 1  -  x  ' 

£  12 

8rOUP'  any  °rder  2  prediCi“  “»  on  ,  £igure..  ,.dl££ec<mce. 

vector  spectrum."  defined  >o 

the  sequence  of  the  nunber,  „f  pait.  o£ 
separated  b,  each  angle  distance  pair.  •  The  t«o  ftgures: 


have  the  sane  difference-vector  epectra,  i.,.. 


"vector" 


number  of  pairs 


M 

Br 


% 


1 
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) 


Hence  no  order  2 nroH 

edicat  e  can  make  a  clasaification  which  is  both  transit 
"ant  and  separates  thcae  two  figures.  In  facr 

-  «-  Stoop  invariance  theotM  is:  C°"‘~ 

^  -  *00  W  .  translation- invariant  predicate  of  order  ,  ^ 
n(v)  to  be  the  number  of  paira  o£  poi„ts  in  -  . 

v  Then  e/va  are  SePara“1  <>y  the  vector 

_*  Then  y(X)  can  be  written 


*  ~  r  La  n(v)  >  0  T. 


9(v)  are  satisfied. 


oy  any  translation  of 


££2£f:  n(v)  predicate*  in  the  class 
X. 

Xwo  figures  with  the  same  translation  spectr™  n(v,  ca^ot  be 
gu  S  ed  by  a  translation-invariant  order  2  perceptron. 

SSSEi£:  the  figures 


are  indistinguishable,  while 


and 


e'8-  nr(Vl><A)  <n<V«>  -  *. 

— -  -  zsr  <  n(vw)i-  - 
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m 


8$ 


and 


m 


have  different  difference-vector  spectra,  and  can  be  separated. 

If  we  ad i  the  requiraient  of  invariance  under  rotation,  the  last  pair 
above  becomes  distinguishable,  because  the  equivalence-classes  now  group 
together  .11  differences  of  the  sane  length,  whatever  their  orientation*. 

An  interesting  pair  of  figures  rotationally  distinct,  but  still  indistinguish¬ 
able,  for  k  *  l,.  is  the  pair 


and 


which  have  the  t*tme  (direction-independent)  distance-between-point-pair  spectra 
through  order  2 *  namely: 


Mote  that  vc  didnotallov  reflection.,  yet  these  reflectionslly  opposite 
flgote.  ere  «v  confused.'  One  should  be  cautious  about  usin^'intuit^"  £fre 
“Wlonnl  inv.tl.nce  requite,  careful  attention  io  tiie  effect 
of  the  discrete  retipsl  approximation,  hut  can  preswably  be  made  ronalstent 
by  eppllcetlot  2<W.  netted.;  for  the  dlUtetion  "group,"  th«e““ 

serious  difficulties.  There  is  of  course  no  difficulty  for  the  90°  rotations 
the  only  rotation  group  used  here.  rotations, 


9- 


4* 


lx^  ~  *j|:  ■  1  from  4  pairs 
iXi  “  xj  I  "  A  from  2  pairs 
!Xi  '  Xjl  *  2  from  2  pairs 
Ixi  ~  x j  1  “  *^5  from  2  pairs 

and  each  has  5  points  (the  order  1  spectrua). 

The  group-invariance  theorem,  §2.3,  tells  us  that  any  group- invariant 
perceptron  can  not  distinguish  between  members  of  equivalence 
classes  of  masks,  but  can  depend  only  on  the  pattern's  "occupancy  numbers," 
i.e. ,  exactly  the  "geometric  spectra"  discussed  here.  Many  other  proposals 
for  "pattern-recognition  machines"— not  perceptrons,  and  accordingly  not 
representable  simply  as  lineer  forms — might  also  be  better  understood  after 
exploration  of  their  relation  to  the  theory  of  these  geometric  spectra.  But 
it  seans  less  likely  that  this  kind  of  analysis  would  bring  a  great  deal  to  the 
study  of  the  more  "descriptive"  or,  as  they  are  sometimes  called,  "syntactic" 
scene-analysis  systems  that  the  authors  secretly  advocate  (Chapter  13). 

6.3  -  cterns  of  Order  3 


6.3.1  Convexity 

A  particularly  interesting  predicate  is 


*CONm<X)  *  ^  *■*  *  solid  convex  figure!. 


That  this  is  of  order  £  3  can  be  seen  from  the  definition  of  ^’convex":  X  is 
convex  if  and  only  if  any  line-segment  whose  end  points  are  in  X  lies  entirely 
within  X.  Given  a  suitable  definition  of  tolerance  approximation,  it  follows 
that  X  is  a  convex  if  and  only  if 

a«X,  bcX  ■■>  midpoint  ([a,b'])cX 

hence 

“  T  E  («eX^b«X^mid<[a,b])^xl  <  ll 

is  of  order  *  3,  and  presumably  of  order  -  3.  This  is  a  "conjunctively  local- 
condition  of  the  kind  discussed  in  §0.2.  Note  that  if  a  connected  figure  is 

not  convex  one  can  further  conclude  that  it  has  at  least  one  "local"  concavity, 
as  in 


or 


with  the  three  points  arbitrarily  close  together.  Thus,  if  we  are  given  that 
X  is  connected,  then  convexity  is  also  diameter-limited  order  3. 


^  **  *re  sure  X  is  connected ,  then  the  preceding  argument  ftili  in 
the  diameter-limited  case  because  a  pair  of  convex  figures,  widely  separated, 
will  be  accepted: 


Indeed,  convexity  is  probably  not  order  3  diameter-limited,  but  ons  should 
not  jump  to  the  conclusion  that  it  is  not  diameter-limited  of  any  order, 
because  of  the  following  "practical"  consideration; 

Even  given  that  a  figure  is  connected,  its  "convexity"  can  be  defined 
only  relative  to  a  precision  of  tolerance.  If  this  precision  is  not  infinite, 
and  it  cannot  be  in  the  diameter-limited  case,  then  either  there  will  be  a 
bound  on  the  size  of  the  acceptable  figures,  or  some  small  negative  curvature 
have  to  be  tolerated.  But  within  this  constraint,  one  can  approximate 
an  estimate  of  curvature,  and  define  "convex"  to  be  J  curvature  £  4n.  We 
will  discuss  this  further  in  §8.3. 

6.3.2  Rectangles 

o 

Within  our  square-array  formulation  of  E  -  R  we  can  define  with  order  3 
the  set  of  solid  axis-parallel  rectangles.  This  can  even  be  done  with 
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1 


•*> 


diameter -limited  9*5,  by 


rs  «p__  +  x 


F 


s  4l 


-kere  all  »•.  equivalent  under  90°  rotation  are  Include.  The  hollo- 
rectangles  are  caught  by 

r2E  <*W»  +  2  'f’n  *  12"! 

tr  bP 

-here  the  coefficient,  are  eho.en  to  exclude  the  case  of  two  or  nor.  eepar.te 
polnta.  These  example.  ere  adnlttedl,  weakened  by  their  dependence  on  the 
chosen  square  lattice,  but  they  have  an  underlying  v.lidlty  in  th,t  t„, 

In  question  ere  definable  In  tame  of  being  rectilinear  with  no  nor.  then  four 
corners,  and  we  will  dlseus.  this  slightly  wore  than  "conjunctlvely-local" 
kind  of  definition  in  Chapter  8, 

One  would  suppo.e  thet  the  sets  of  hollow  and  soXld  squares  would  h.«  to 
be  of  order  4  or  higher,  bec.u.e  ,.he  co.parlson  of  slde-lengths  should  require 
at  leest  thet.  It  Is  sur»rt.lng,  therefore,  to  find  thet  they  hay.  order  3. 

The  construction  t.  distinctly  not  conjunct ively- local,  w,  u 

to  Chapter  7,  even  though  It  satisfies  the  bounded-coefficient  stipulation  for 
the  present  chapter. 


Another  example  of  an  order  3  predicate  is 

TX  lies  within  a  line  and  has  s  n  segments! 


It 


r 


which  can  be  defined, 


up  to  a  tolerance,  by 


^  ^83  +  E  Z  ^  *  nE  (a11  non-coliinear  triples)  s  nl; 

6*3*3  Higher-order  Translation  Spectra 

If  we  define  the  3 -vector  spectrum  of  e  figure  to  be  the  set  of  numbers 

of  three-point  rn.sk.  s.ttsfled  in  e.ch  translation  eqy  elence-cUss.  it  i. 

interesting  to  note  the  following  fact  (which  is  about  geometry,  .nd  not  .bout 
linear  separation). 

Iheorm  6.3.3:  Figures  are  uniquely  characterised  (up  to  translation)  by  their 
3-vector  spectra,  evui  in  higher  dimensions- 

Proof:  Let  F  be  a  particular  figure.  The  figure  F  has  a  -diameter";  the 
maximal  distance  between  two  of  its  points,  choose  a  pair  (a,b)  of  point, 
of  F  with  this  distance  and  consider  the  set  ^  of  Msk|  of  ordw 

3  that  contain  a,b,  and  any  other  point  x  of  F.  These  masks  must  have 
coefficients  equal  to  unity  in  the  translation  spectrum  of  F,  for  if  3? 
contained  two  translation- equivalent  masks 


then  one  of  the  dist.nce  te.gb]  or  tg«.b)  would  eacced  D,  for  they  .re 
diagonals  of  a  parallelogram  with  one  side  equal  to  D. 


Thus  any  translate  of  F  must  contain  ajmisue  pair  parallel  to  (a,b>.  and 

P”“  °f  “5  SPeCtr,m  allows  one  so  reconstruct 

completely  the  figure. 

X''  - _ 

» *  * 


>  ✓  ok  ✓  •**  * 

/  *1  *  ^  - 
/'iv 


The  fact  that  a  ftgurc  t.  determine^  its  3-translation  spcctriw  j 

°f  l"P4'  *“  -  «U.~.  of  figures  ls  ord„  3.  (It 

dows  imply  that  the  translations  of  two  different  „ 

,  .  reiK  figures  can  be  so  separated. 

.n  tact,  the  method  of  §7  3  Annn„„fc, 

w.3.  Application  , ,  shows  this  can  b.  done  with  order  2 

but  only  „utslda  th.  bounded-coefficient  restriction.) 

y  f  the  relation  between  spectra  and  ordinary  geometric  concepts 

/  Che  d°"*ln  °*  *******  •  Sl|bject  with  which  we  are  not 

°"e  W°"ld  fer  —PI*,  bow  much  of  the  snectrum  is 

necessary  to  ch.r.cterif.  figures  invarlantly  under  X-^-and  what  percepfrOn 

“Pl0lt  tMa  ^  to.,.,.  the  porceptron 

'  may  be  higher  than,  the  spectral  discrimination  order.)  Xhe  remark. 

about  rotation,  in  Chapter  7,  suggest  that  v, 

suggest  that  the  problems  about  spectra  under 

rotation  are  quite  a.  bit  deeper. 


6,A  Patterns  of  Order  4  and  m^w 


A3  -own  i„  to.,,  w.  can  use  the  fact  that  any  three  points  determine  a 


9 
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circle  to  make  an  order  4  perception  for  J»e  predicate; 


lx  is  the  perimeter  of  a  complete  circle! 


by  using,  che  form 


r  £2  xx.  x  x ,  +  r’* 
d/C„,.  a  b  c  d  ht 


a  be 


xaxbVd  <  11 


abc 


L 


wtjere  Cflj)c  is  the  circle*  through  x  ,  x  ,  and  x  . 

o  D  C 

Many  other  curious  and  interesting  predicates  can  be  shown  by  similar 
argumchfc*  to  udve  small  orders.  On  should  e  careful  not  to  conclude  that 
this:  means  that  there  are  practical  consequences  of  this,  unless  one  is 
prepared  to  face  the  fact  that 
a) 


b) 


c) 


large  numbers  of  ip’e  may  be  required,  of  the  order  of  lRlk"1 
for  the  examples  glv  }i  above;  ,R| 

The  threshold  conditions  or*  sharp,  so  that  engineering 
considerations  may  cause  difficulties  in  realizing  the 
linear  summation,  especially  if  there  is  any  problem  of 
noise.  Even  with  simple  square-root  noise,  for  k  *  3  or 

Th«8Cr,cSe, 00130  erows  £aster  than  the  retinal  size. 

in  ClmptSC9?nt  *iZe*  ar<!  °ften  large'  a*  8hown 

*  very  slight  change  in  the  pattern  definition^*  often 

0r^r°f  fecognizability.  With  low  orders, 
it.  may  not  be  possible  to  define  tolerances  for 
reasonable  performance. 


* 


retina?”  faTfaV  toUtanc<i  probi0,,:  what  a  ci^ie  in  the  discrete 

'k*k 

See  note  at  end  of  Chapter  0, 


*  • 


— 


-/> « 
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6-5  -°~gtr‘l  -n, - 

thMr“*-  “•  w*«  of  th°  rouo‘'1"8 

ne  gfoup  Invariance  theory  <§2  2>  .h« 

wr**/  ahows  that  if  a 

invariant  with  remote  to  .  Predicate  4  1, 

aspect  to  a  group  c  thm  „  ^ 

realized  by  a  form  can  fce 


f 


*  "  ff  »kHi  «>1 

vtiera  the  are  the  number*  c£  aj'a  ln  each  r 

by  X.  I„  {5  2  we  t  a  «-«1“l*nlence  cl...  ..tis£Ie<i 

40  10,2  we  toucned  on  the 

figure. -under  the  group  of  tr.„.,.tlo„.  .ZlTuUn^  ^ 

fact  the  N  (X)  number  *  These  spectra  are  in 

"t{X)  numb<er*  up  to  order  k  -  2  if  .  *,  * 
de.crlbed  by  a„y  condlt,  ‘  '-‘"'“rl.nt  *  cannot  be 

y  _SX  condition  on  the  N  'a  for  a  given  *  m. 

In  «♦).  Ih,  fol lotting  r«.„lt.  ho-  ’  °  ly  *  is  “>‘ 

*  .  “Its  show  some  conditions  on  the  K  thai-  <  , 

t  is  of  finite  order.  Wi  ttot  iaP1y  that 

PPose  that  f  1*  defined  by  slaul%aneou(P  gati*facti 

«“s  sati.factlon  of  «  equalities: 


KX)  a  f  Ml(X)  -  nL  and  N 


(X) 


n2  and  —  Hm(X)  »  „1 


ID 


”l'  *"»  nB  is  a  finite  sequence  of  integer*  Then  *  h 

xiaegeri,  Then  f  has  £  twice  h,*. 

;r,;r 


T 


n 


!  . 

;  f 

Theor—  6.5.1,  Let 


*  *  *1  U  *2  U  —  U 


“  Ito  <P**£  And  *p(X)  -  l}| 


E  <HX>. 
*“i 


Then  the  order  of 


*<«  -  r»l(X)  f„  <1Sl1 


i*  at  aost  twice 


Ih.  *o.l  .£  the  proof  t.  ro  .ho.  thht  the  definition  of  *  «,  he 
in  the  for.  of  .  Ilneer  thre.hold  «*pr.„ion,  vir.; 


tOC) 


r  z  (*!<*)  -  nt)2  <  l  1. 


A.  it  «.nd.  thi.  i.  not  .  linear  thr.rtold  oo*io.tiou  of  predict...  I. 
recaet  it  into  th,  d..ired  .hep.  introduce  .d  hoc  convention. 
th.t  .ill  not  he  „.ed  «l.«here.  Given  .ny  „t  #  o£  ^ 


construct 
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a  aew  *et  of  predicates  4<2> 
predicates  in  *  .nd  defining 


*>y  first  listing  ail 


P*irs  of  (9^)  of 


A  9. (X) . 


Many  of  the  predicates 
example: 


80  constructed  will  be 


logically  equivalent. 


for 


but  we  make  the  convention  that  the*. 

of  ‘<2)-  Oku  ^  .  “*  t0  “  — -  “  neabers 

*  th*t  *  *  ™ry  strict  sense  i<2>  , 

ferns"  rather  than  of  predicates.)  *  *“  °f  “>>r‘,dic«< 

«  effect  of  the  convention  is  to  slnpUfy  thl.  arith.eti 

'  «  «e  about  to  L.t  -  .  l0*iC 

««ctly  H  predicates  In  #  ,c.  Mtiift-  ,*  “  fl8“Ce  f°r  ,hlch 

•fled.  Obviously  N2  nrnjj  (2\ 

will  be  satisfied  by  X,  i.e.,  Plicate,  of  *<2> 


E  <P(3P  *  N2. 


Now  let 

1**2 

Che  number  of 


pt*dlcat,s  •*  *i  -Msflal  by  x  1, 


-i^vaience  classes 


Ni(X)  *  £  <P(X); 

9^ 


Since 
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then,  as  we  have  seen> 


Thus 


<P(X) 


Ni2(X). 


E 

i 


{ 


<pm 


-  2  n  E  <P(X)  +  n  2} 

9***  1  J 


?  { (**«  -  0*  }• 


To  represent  the  left  hand 
linear  threshold  predicates 
The  linear  form  we  want  is 


*ide  of  this  equation  in  the  standard  for  for 

we  define  ♦'  »  n  *  -ii  ft-u- 

lthe  constant  function}. 


where 


and  then 


£  a(<p)  9 

a(9)  *  1  for  9«*(2> 

0(9)  *  -2n^  for  9®#^ 
a(constant)  -  E  n.2. 


*(X)  «  rzh:(9)9<x)  <  ll. 

To  complete  the  proof  of  the  theorem  we  have  only  to  observe  that 

•s(<J,i,j)l  *  Npp  us<Vj)| 

<  Isop^l  +  js(9j)  | 

<  2 (Max  |S(9)|;  9*#). 


Q.E.D. 


I 


6*5*2  Sxtgnded  Exact  Matching 

*"  obvious  g««t«lu,tioi>  of  Theoree  6.5.1  t.  thtc; 
Suppose  that  $  is  defined  by 


n  m 


♦(X)  =  V  A  (N  *  n  ), 
i-1  j=i  i  ij'* 


i.-e  ,  *  satisfies  any  one  of  a  number  of  exact 

^act  conditions  on  the  N  .  Then 

is  of  fioit.  order.  £or  ve  .  re.Ua.  the  p,^mUl 


n  a 

r  s  s 


i-1  a  <¥»  -  ”ij>  <  n 

methods  like  those  in  the  previous  paragraph.  However,  the  extensi 

«,ui«.  B001..0  producti  of  pcrttcatM  of  diff#rOTt  •  h«  « 

the  maxiaal  order  retired  yiUbaig-  ”  |s  |  CU““’ 

associated  with  1^.  k"1  k  PP°rt  8iz* 

6,5,3  Kean  Square  Variate 

Instead  of  the  predicates  discussed  in  §5  5  , 
higher  values:  ta«“«  6  « 


r^oij  -  Uj)2  <  ei. 

Thee  the  ayate.  will  accpat  flgllt„  for  *,uh  ^  ^  ^ 

differences  of  the  N  ’s  and  tK  i  - - SaU*r**  of  the 

he  a  end  the  e^'s  are  bounded  by  6,  *ey  patter„. 

cias.iflc.tion  -Ohio,  via  b.  a„.itiv.  to  certain  „l„ds  f  „ 

Ms  observation  hint,  that  it  ,i,ht  be  u.eful  to  st  ”  0"’  *”d 

udy  such  machines,  and 


perceptrons  in  particular,  in  terms  of  their  spectrcm-distortion  characteristics. 
Unfortunately  we  don't  have  any  good  ideas  concerning  the  geometric  meaning 
of  such  distortions.  The  geometric  nature  of  this  sort  of  "invariant  noise" 
is  an  interesting  subject  for  speculation,  but  we  have  not  investigated  it. 

6.6  Figures  in  Context 

For  practical  and  theoretical  reasons  it  is  interesting  to  study  the 
recognition  of  figures  "in  context,"  for  example: 


tOO  =  fa  subset  of  X  is  a  square!, 


t(X)  =  Ha  connected  component  of  X  is  a  square!, 
or  even,  to  begin  to  consider  three  dimensional  projection  problems: 


*(X)  -  Tx  contains  a  significant  portion  of  the 
outline  of  a  partially-obscured  square!. 
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Th®  show  that  there  is  more  than  one  natural  meaning  one  could 

give  to  the  intuitive  project  of  recognizing  instances  of  patterns  embedded  in 
•contexts. >•  We  do  not  know  any  general  definition  that  might  cover  all  natural 
senses,  and  are  therefore  unable  to  state  sharp  theorems.  We  do,  nevertheless, 
claim  that  the  general  rule  is  for  low  order  predicates  to  lose  their 

property  of  finite  order  when  embedded  in  context  in  any  natural  way.  To 
illustrate  the  thesis  we  shall  pick  a  particularly  cormnon  and  apparently 
harmless  interpretation: 

^in  context ^  =  for  component,  Y,  of  xl. 


It  will  be  obvious  that  the  techniques  we  use  can  be  adapted  trivally  to  many 
other  definitions. 

Intuitively,  we  would  expect  *in  tc  be  much  herder  for  e  percept™ 

since  the  context  acts  as  noise  end  the  parallel'  operation  of  the  device  allow, 
little  chance  for  this  to  be  separated  from  the  essential  component.  The  point 
appears  particularly  clearly  in  the  cases  where  »  uses  rejection  rules.  These 
cannot  be  transferred  over  to  *in  for  very  obvious  reasons.  Similarly, 

we  loose  the  stratification  methods  of  Chapter  7  and,  indeed,  most  of  bur 
technical  tricks  used  to  obtain  low  order  representations  of  predicates. 

The  next  two  theorems  show  how  this  intuitive  idea  can  be  given  a  rigorous 
form.  It  should,  however,  be  observed  that  no  simple  generalization  1.  po.alble 
bout  the  relation  of  t  to  context  since  some  t's  become  degenerate  in 

context.  For  example,  every,  set  has  a  connected  subset  of  odd  parity  and  every 
set  has  a  connected  component! 
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5SS£SLL!  I**1  E  be  a  finite  square  retina  and  let  f(X)  be 

♦(JO  •  Tl  is  exactly  one  horizontal  line 
across  the  retinatl. 

Then  *  i.  of  order  2  but  ^  i,  not  of  finite  order. 

Proof:  We  leave  an  an  exercise  the  proof  that  ♦  as  defined  has  order  2.  To 
show  that  context  is  not  of  finite  order  we  merely  observe  that  it  is  the 
negation  of  the  negative  the  one-in-a-box  predicate,  f  Let  G  be  the 
“  X  ®  arr*y  of  unshaded  points  discussed  in  §5.1.  Taking  this  as  our  retina, 
the  predicate  ♦  l  asserts  that  there  is  not  horizontal  white  line  across  the 
retina.  Its  negative,  in  the  sense  of  §1.7,  asserts  that  there  is  no 
horizontal  black  line.  Since  ♦  x  is  not  of  finite  order,  the  argument  of 
§1.7  shows  that  the  saae  is  true  of  its  negative.  And  by  reversing  the 
predicate's  inequality  we  find  the  saae  is  true  for  the  desired 

♦in  context  "  **  contain*  ■  horizontal  line  across 
the  retinal. 

Theorea  2;  Let  ♦(!)  be 


fr  Is  a  hollow  squarel. 


Then  ♦in  context  iM  not  o£  finlte  order- 


x 


□  rj  □  n 


a 


i*2o£:  The  proof  is  exactly  the  „  . 

—  - ««  _  ~  — *“  “• 

.  * — ^  lt  M  ta  .1  --  ~  ~ — - 

in  tM.  case,  „rder  3.  **  °f  — «• 

~ !  *■  *Uer“ttVe  —  *•  «  i»U  «*  Uee.  of  «tc„.  . 

"  ““  •—  — ««-  in,  co„ec„nty(fc.3,. 


b 


— jfMMMJZAIlOM  AMD  STIATinCATTOM 
7*1  gqulVlOC*  Of  rigurmm 

In  previous  chepter.  vs  discussed  the  recognition  of  pstterns-clesses 
Of  figures— closed  under  the  trsn.fom.tlon.  of  group.  V,  no.  turn  to 
th,  related  question  of  recognising  the  aqulualonca.  under  .  group,  of  an 
arbitrary  air  of  figures.  He  remit,  belou  ver.  surprising  to  us.  for  ». 
had  supposed  that  such  probl.s  ver.  not  generally  .f  ffotte-order.  A  nmber 
of  questions  ramin  open,  and  the  superflci.il,  positive  character  .f  the 
following  const  ructions  are  clouded  by  th.  apparently  enomous  coefficient, 

they  TtVltn-  ln  «•>**  '**  lncros.e  «it„  the  ala,  of  the  retina. 

A  typical  problem  has  this  fon:  The  retina 


1.  presented  a.  too  equal  part,  A  and  B  and  «  a.k:  i.  the  figure  In  part  B 
a  rigid  translation  of  the  figure  In  part  A!  More  generally,  i,  there 
elenent  g  fron  sour  gtvan  group  G  of  tranafomatlon.  for  vhich  l  1,  tha  remit 
of  g  operating  on  A?  Hhat  order  predicate,  are  require  to  oak,  such 
distinctions,  The  remit,  of  this  chapter  all  derive  fro.  of  .  technique 
*o  =.11  stratification.  Stratification  mka.  tt  posaible,  under  certain 
conditions,  to  alnulata  a  s.qumtUl  proc...  by  a  parallel  process.  ,a  vhtch 
*  of  this  Chapter  spdIv  direct! * 

retinas;  that  is,  without  having  to  consider  liaiting^rocesHe^on^e"51'11110 
of  finite  retinas.  8  processes  on  sequences 


the  results  sre  so  weight*,  that,  If  certain  condition.  ,r«  satisfied,  so.e 
computations  will  numerically  outweigh  effects  of  others.  The  technique 
derives  from  the  following  theorem: 

Theorea  7.2;  (Stratification  Theory) 

n  ^  f  « 

I’  S2s  '*'»  *j*  a  sequence  of  predicates  and  define 

a  sequence  ....  Cjl  ...  of  classes  by 


XCCj  A  (k  >  j  —  >  ~ 


Thus  X  is  in  Cj  if  j  is  the  last  index  for  which  ^(X)  is  trues 

let  4  =  {»t}  be  a  family  of  predicates  and  let  ij . *  ,  ...  „ 

ordered  sequence  of  predicates  in  1(4)  that  are  each  hounded  in  the  ...£ 

for  each  i.  there  Is  a  linear  for.  with  Integer  coefficients 
^ j  ~  j  ~  9  such  that 


*i  '  rV<n 

and  that  there  ealst  bounds  Bj  such  th.t  lEjtX))  <  Bj  for  all  finite  |x|* 
Then  the  predicate  ,«)  -  T  XcCj  ->  *j(X)  1  obtained  by  cahing,  on  each  C  , 
the  values  of  the  corresponding  lies  in  l(M).  that  is,  „  be  wWJ 
3S  a  form 

ooJunifr!y?Ct“aUy  re<|UlreS  °"ly  th«  lEj»)l  be  funded  on  each  c^,  i.e.. 


■x 
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f(x)  «  l  s  (hj  a 


I 


Th«  partition  into  Cj  (X). 
Fif\  7,2,1 


Usually  it  will  be  the  case  that  for  any  finite  jxj ,  X  will  Ho  in  on* 
of  the  Cj.  otherwise  we  will  be  interested  only  in  the  values  of  KX)  foe 
XcUc.. 

J  J 

The  proof  is  by  an  inductive  construction.  Define 


and 


st  -  *1 ' 

|r  | 

I SJ  ’  Vi '  «A  +  <»j +  l)-vV 


a 


T,*e  bounds  B, 

j  he  *-'*l*tence  of  the  M  «s  v 

•■*“«*  lw  ttU  taltatt.  J  '  *«-l  «. 

s,!Vv\ 

""‘1  WU  .hov  that  m  .  ,  m  0  . 

•■«»««,  Mm  ,»«,  x  u  „  s«  *•  "U^BW 

HX):  **  i  s  (x)  ^  q  i  *t  if  X  is  in  q  the- 

l«  0  1.  INDUCTION!  Ass!*.  th.t  .  1  *  !»  M 

,W  *  '  SM«  "  0  I.  Nov  the  cocfflcl  "  °M  th°n 

<m<l  s  "t0  SO  if  XsCj ,  ^  ,  , 

U  U  ,(1°  """  SJ  4  l  so  S  i  -M  -  M  +  >!,  + 

u)  if-*«  a«^«o  J,  ‘  J 

J  v  «  0. 

The  order  of  *(*) 

JeS™e  in  i  end  the  mxim  "°  *****  thftU  the  SlEa  w  f;"  u,, 

“*imu™  support  in  n.  rhl,  ,  ‘ 

in  *  follow,  because 

■I  o«ur  o„ly  Ui| 

g  QCs  Klth  P*«Uo,eo» 

«>US  the  construction  aasu««s  chab  , 

*  its  intersections  vith  th,  ,W  ""  *  “«  -o 

r*—  i" *  -uo.ti.ns  Mowt .  ^ . . 

,M*‘  *  d—  deviations  of  a  ^  -  " 

8  ftQm  3  >»U"  ,  e8Ulo|, 
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cloee  connection  between  the  poasibiUty  of  ct.i.HtrMctiuK 

;rr;;r  •* t,,u  — *  >«-*  *-».*« . . 

identifying  a  figure  first  by  normal***™  . 

normalized  image  with  .  protoeyjJ -  “  ^  C°"P*r1"8  « 

.in  :r t,j  ■“* !hat  •*—  *  - «.  01  mi.  th.r„ 

enormous  coefficients,  growing  exponentially  ,or  f.eter  with  the 

stratification  Index  J.  Ihu.  the  result,  of  thl.  h 

of  thi*  chapter  should  not  be 
considered  o£  practical  interest.  They  are  inr*  rf 

.  ■  y  **  **  of  theoretical  interest  in 

rr—  reUt‘°n-0t  P~»'  non-relation-'of  the  structure 

.o  transformation  group.  to  the  order  of  certain  plicate.  im.rl.„t 
under  those  groups. 

7-3  1=  Symmetry  .1^ - 

*-  *  ■  V  -  b«  the  point.  „f  infinite  linear  retin.,  1... 

09  <  X  <  00.  «»  x.t.  , 

2> 


*3  S\X-I  X»  *i  \  Z3 


Suppose  that  X  is  a  figure  in  *  with  finite  |x|. 


We  ask  whether  the  predicate 


•  r  *  It  symnetrlcsl  under  reflection*  1 

advScef'Sr^  o'o"J  "nt"  “  not  specified  in 

on,  Slffidnlti,  wlth  thi^n^  -d«  he, 


i*  of  finite  order. 


* 1,111  ♦« *  v  «  v  ...  eh.t 

“  10  ““  ')“*try'  -*•  th*  Ml-W  trick:  th«  „  ■.  „U1 

find"  tho  two  "and  do  lata”  a#  ^ 

"apo“t'  ot  *  *od  «*ch  of  the  *  fi  will  *-*+  +u 

lie  s  will  test  the  synmfetry 

°  '  fl*“*  &,i  “*  “"*•»•-»«  Wit  ol  -Wpolnti.  our  then,  t, 

to  define  the  ^  •  *>  that  met,  C  -  will  be  the  class  of  figures  with  a  certain 

pair  of  endpoints.  To  do  this  we  ne*4  *  *  v 

’i'  *•*  to  be  *n  enumeration  of  all 

••pwnts  for  sny  s  and  for  anv  >  n  . 

0  sfd  7  .  tor  any  d  i  0,  sith  the  property  that  any 

t*ra  (*,»**«>  mist  follow  sny  ten  (x  .  ,x  )  with  0  -  a  <  a.  v 
•  sfs*  sfefb-  4,1  th  0  *  a  £  a  +  b^d. 

*  ***  «*  «*“««.  «  ta  to  Pig.  7.2.1-!. 


il>  »  -mm  1.  0T.T  follomd  tnoth.r  tta,  It..  wthln  lt. 


—  -  A 
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Therefore,  if  x^x^  sre  the  extreme  left  and  right  points  of  X,  then 

X  will  lie  in  precisely  the  ^  for  that  (x^x^).  Mow  define  *  to  be 

«# 

”  r  Xa+i  *  x»fd-i*  1  “  °*  «*  1 

or*  equivalently, 

*j  *  [J  (W<1  •  W  «  o  t 

showing  that  it  is  a  predicate  of  order  2  which  is  bounded,  with  B  <  - 

j  2  ’ 

So,  finally,  application  of  the  stratification  theorem  shows  that 

+SYM  °rder  *  4»  *ince  the  *'a  have  order  *2  and  the  Vs  have  support  $Z. 

7.4  Application  2:  Translation-Congruence  along  a  Line 

Let  ...,  xai  ...  and  yt,  ...  be  the  points  of  two  infinite  linear 

r6tinfl8|  j j  •  ®  ^  y  ^  x  {p. 

s  J  t  * 

U*l  I  mjmp  rr urn  vm  r> 

xa  V 

Let  X  be  a  figure  composed  of  a  part  XA  in  the  left  retina  and  a  part  X*  in 

the  right  retina.  We  want  to  construct  fc^CX)  -  fthe  (finite)  pattern  in  A 

is  a  translate  of  the 

^  h  x  i  _  pattern  in  Bl# 

To  *t ratify  iTMSS  we  have  to  find  a  eequence  itj  that  allovs  ua  to 

test,  with  appropriate  ^'e,  whether  the  A  end  B  pert,  of  I  are  congruent. 

We  will  do  this  hy  a  nethod  like  that  ua*d  in  -J 7.2.1  hot  wa  have  now  to  handle 


*^****  *^**^^**OMly.  Hut  w.  . 

__  *  We  need  •  sequence  of  n  s  thmt- 

an  quadruple.  in  muchM  J  th4t 

only  if  the  -*,1*.  o£  Its  A  and  g  ‘  ^  ^  ^ 

values  of  x  ,  x  precisely  the  corresponding 

*  *****  **  *"*  yt*dy*  0»are  does  indeed  exist 

—  —  be  obtained  f ro.  th7x  •.  of  #721  ‘  «' 

try  to  find  one  Massif)  *  *  foll°vs  (the  reader  might 

^  *jk  to  W  th.  four-point  ns.k  obtained  fay 

"jk0®  *  *j(V  •  xk(^), 

that  is,  by  choosing  accordlnc  i  ^ 

Point,  0£  *.  Tfc.  ___  '  P°1”t‘  °f  A  rad  accor<iing  to  j  t» 

""•t-®  oesoence  rt, . 


if  naftim 

noquance  requires  us  to  enunerate  an  , 

the  condition  that  n  11  *n  s  ul‘^er 

90  *ab  **“  Precede  any  *  if  v,...  _  3 

a  «  cd  hoth  a  ^  c  and  bin 

A  solution  is:  d* 


f 


*11 J  *21’  *12*  *22*  *31*  *32*  *13’  *23*  *33;  *41*  *42’  *43*  *14’  *24*  *'*  r 

and  for  the  *.  tern  in  thia  sequence,  an  appropriate  predicate  .  is: 

(jll) 

♦(Jk)  •  Tthe  segments  defined  by  and  ^  have 
the  sane  lengths,  and  the  x’s  and  y'a 
in  those  intervals  have  the  sane  values 
at  corresponding  point si. 

This  is  an  order  2  predicate,  and  bounded  (by  the  segnent  lengths).  The 
jtj '  s  now  have  support  4,  so  has  finite  order  s:  6.  Actually,  having 

found  both  axtraaa  of  XA,  it  is  necessary  only  to  find  281  end  of  Xg,  so  a 
slightly  different  construction  using  the  aethod  of  $7.9  shows  that  the  order 

of  *TBABS  l*  *  5* 


^•5  Application  3.  Traualation  on  the  Plm» 

*•  “th°"  °f  appacatl°n  2  -  «PP»«0  to  tho  prob.o  oi  tho  two- 

“f  *  *—  ~  cf  the  plane  by  usl„6  the 

following  trick: 

^  C°Py  °f  ^  mlM  b*  “  (»*  •>  «*.  Arrange  the  scares  into 

a  sequence  {x  }  with  the  square  at  (a  hi  4  . 

i  quare  at  (a.bj  having  index  ma  +  b.  In  effect  we 

Creat  ****  retina  as  a  cylinder  mod  index  its  squares  so: 

,*  *  *  •  «  -  -  ^ 

t|3insi±Tiw 

(f* - $ 

,V> _ - 12  £2  'll 


This  maps  each  half  of  the  retina  , 

the  retina  onto  .  lip.  llke  that  of  appUcatimi  2 

in  such  a  way  that  for  limit.*  ^,nai,M _ _  , 

.  ■  ‘ - ~  1  tll,r  d°  not  °otry_the_fiffl;.e  y  over 

the  edge  of  the  r«Hn»  translations,  on  ►»,«  i 

T~7~  “  the  I’l*M  are  equivalent  to  translations 

ne*  “d  “  °rd*r"5  predlcafce  ca"  be  constructed.  in  $  7.6  we  will 
Shoe  ho.  the  ugly  reetrlction  Juet  iapo.ed  c.„  be  ell*lnateiJ| 

^.tiqo  4.  UP-  rot>r1n,  ,  „  ,  , 

With  the  .«  hind  o£  restriction,  thi.  predicate  can  he  cocatrncted  („ith 

orders  fro.  application  1  hy  the  ....  rout,  that  derived  application  3  iron 

application  3.  SUilarly,  ..  can  detect  reflections  about  arbitrary  vertical  ones. 
7.6  Repeated  Stratification. 

In  the  condition,  of  the  Stratification  Theoren,  the  only  restriction  on  the 
.  is  that  they  b.  suitably  bounded.  I„  certain  applications,  there  is  no 


reason  the  fj'*  themselves  cannot  be  obtained  by  stratification.  This  i. 
particularly  easy  to  do  when  the  support  of  ^  is  finite,  for  then  boundedness 
is  immediate.  To  iUustrate  this  repeated  stratification  we  will  proceed  to 
remove  the  finite  restriction  in  Application  3  of f 7. 5. 

First  enumerate  all  the  points  of  each  of  two  infinite  plane  retinas  A  and  3 
according  to  the  more  or  less  arbitrary  pattern: 


■■■nrcznummB 

BiimiGiiMnsi 

■■■imsaunaS 


to  obtain  two  sequences  . ...  ,nd  ^ . ^ 

had  *  ,1“11*r  problem  there,  only  noy  it  i.  ,  tvo-diMn.ion.1  version. 
Nov  we  will  invoke  precieely  the  oration  a.  i„  7.4.  but 


'jk  « 


<VXA*»k'V  *Vk 


Then  CyR)  i«  the  clae.  o f  pair,  of  figure,  whoa,  highest  (in  the  x-em.er.tion> 
point  of  XA  is  and  highest  point  of  Xg  is  y^ 


“■"Riir  i 


An 


We  need  only  a  (bounded)  *  h,*-  ,  .  , 

,  .  <*>  h“t  d*Cld“  \  «  translate  „f 

for  figures  In  C.  But  the  «•»»..  4  ~ 

*  "  CJk  011  lle  wltht»  bounded  portions 

the  planes.  In  fact  vithin  equate,  of  about  ,,,'s  on  M 

a  .  on  a  Side  around  the 

riglns!  Within  such  a  square  -  or  better  witM 

to  avoid  "edge-effect,”  W.  “±th  ^ 

Ch.„t  7  ,  *"Ply  direC£^  th*  «‘»lt  of  application  3, 

apter  7.5  to  obtain  a  predicate  4  . . 

with  finite  ^  “attly  the  desired  property,  and 

with  finite  support!  The  requiting  order  la  «  5  +  2  .  7  „ 

, .  .  „  ^  '•  (We  have  another 

s  ightly  fallacious  construction  that  yields  order  l 

n«c  yields  order-4,  so  we  suppose  the  true 

value  to  be  somewhere  in  between)  Th.  . 

).  The  same  arguments  can  be  used  to  lift  the 
restrictions  in  application  4.  Chapter  7.5. 

7'7  S-  in  chi 

«.  di8r„.  .  wonent  to  apply  th.  Mthod  of  the  last  section  to  she,,  that 

e  predicate  *Q(X)  -  X  i.  .  .olid  (hollo.,  a«ls-perallel  square!  (that  is 
ot  the  for.  anywhere  in  the  pian.)  ha.  order  a  3.  ’ 


I  V/*W//t 


(We  consider  this  remarkable  because  inform^ 

t  raal  «8“ents,  to  the  effect  that  two 

sudes  mat  ha  cohered  In  length  *11.  the  interior  is  », 

of  at  Isaac  A  tw  ■  teSI:ed>  su68est  orders 

«.t  4.  Th.  result  was  diecovered  and  proven  by  .nother  nsthod  hy  our 
student,  John  White). 

We  enumerate  the  points  x  nf  a  ,  , 

. . .  a  SI"*le  Piene,  J„st  as  in  1  7.6 

and  simply  set  n.  »  x  .  Then  r  4.  .v. 

}.  1  0}  1.  th,  set  ,lgures  uhose  „hlghest„  point  is 


•V 


I  t 


If  X  is  a  square,  the  situation  i.  llka  on,  of  the  caae.  .hoc, 


we  thee  conattnct  ,,  hy  stratifying  a.  foil™.:  Ut  zj,  ^  ...  *1  „ 

finite  sequence  obtain*  by  .tapping  into  the  .pit.!  figure  orthogoclly  ft™  *  . 

ln*  "4  1  '°  tl>*t  C1  “1U  C°“tain  311  tfc*  ninares  of  length  i  on  a  ai.^ 

fhat  are  ".topped"  hy  Xj.  ,„t  there  is  only  one  anch  square,  celi  it  J  So  to 
complete  the  double  atr.tific.tion  vc  need  only  provide  predict™  pJ  to 
recognize  the  square.  s|.  g„t  this  can  be  done  by  1 

*1  ■rZ*k  '  l2  [\  a  i21 

Vsi  v'si 

Which  is  of  order-1.  < ,  *  has  order  <3» 


Q.EiD. 

7'8  AE£ilCatl°"  6'  and  hi,. . 

can  a  ay.tc  of  finite  order  recognize  equivalence  of  teo  arbitrary  figures 
under  tranalation  and  .ire  change? 


*Kf- 


V 


Some  reflection  about  uhe  reault  and  method  ,  * 

C  and  methods  used  in  f  7.6  and 

7  "iU  th«  “«  hnve  ,11  the 

translation,  and  {  7  ,  ..  „  *  ^  Sh°“*  h°“  “  1“""«  ' 

,  *  show*  how  to  recognise  all  the  tranalation,  and 

:rr- — -  — - 

•  — ...... ......  - . :  :  i-jrr  -  -  - 

„  ,  e  problem  aquarelv 

on.  the  la.,,  ,t  *.  #*«*«*  property  tan  at  lWt  * 

"it"  “““  — -  -  -  ^  noggestive  fashion,  <„  d„ 

ZrZaT  ‘  ~  ~  -  **  *  -  -  of  rotation- 

variance,  because  the  problem  there  u  oi  a  different 

.  *  serene  kind,  that  cannot  be 

::  .  *  -  — e  the  transform  of  a 

nethod.1^  ^  ^  ~  ~  —lo.ll, 

a  0ir  r  UUh  tCChnl,Ue  ~  111  1  7-6  -  «-  predicates 

“'I  ^  ~  «*““  *»  — .  Then,  jUat  as  In  f  ,6,  lhe 

problem  la  reduced  to  finding  predicate.  *  that  a 
=  boxes  of  fig  7  6  1  u  (jk)  °"ly  °**Tat‘  "thin  the 

box^^el^  —  -had: 

*  *"“*ratlon  o£  P°iMs  described  In  Chapter  7.5.  Then  we 
stratify  four  times  („  in  succession  with  reapect  to. 
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and  leftmost  point  of  A 

(2)  y  -  highest  and  leftmost  point  of  B 

(3)  x'  ■  lowest  and  rightmost  point  of  A 

(4)  y'  ■  lowest  and  rightmost  point  of  B 

We  will  need  to  define  predicates  . 

v  icaces  *x>y>x.  y,  for  this.  If  the  two  vectors 

aad  *'  '  y’'  i0  "°e.  h*V*  *“»  direction  Wc  set  p  -  0;  other-la. 

“  need  a  P  to  teat  whether  or  not  for  every  vector  displacer  v 

y  +  J  5  x+fErp-  $ 

and  thl.  1.  an  order-2  predict.,  le«iing  finely  to  total  order  ,  2  +  <  +  2  .  g. 

Of  course,  on  the  discrete  retina  the  Indicated  operation,  on  vector,  „1U  he 

ill-defined,  hot  it  .«.  clear  that  the  result  is  not  v.cuoue:  for - r.r 

"  “k  f°r  reCO«“lti“  <*  the  case  -here  ^  1.  .  trsn.l.te  a* 

nultiple  of  XA  in  ai,.,  »lth  each  black  a,uare  of  XA  supping  into  a  corrcp.nrfl.gly 
larger  square  in  X y 

7,9  -P2llC*tiop  7>  Equivalents  of  a  Particular  - 

In  constructing  p  f„r  application  5,  noted  that  one  c  alway.  .octet 
an  order-1  predict,  to  detect  precisely  one  particular  figure  x,  (by  „.lng 

‘  U  t0ll°“‘  ^  ”  “*  “““net  a  scr.tlficti.n 

(si>  for  a  group  G  auch  that  for  all  geG 

r-CiA  *fcCi  *  <**  -  W  v  (g  -  e) 

then  ««  can  recognize  exactly  the  G-s,ulv.l«nt.  of  a  given  figure  X<)  (uith  on. 
order  higher  than  the  order  used  by  the  stratification  Ihi,  i. 


'1 


suggestive  of  a  «chlne  th>  , 

“  “°"»i  for..  for  thu  Cl„  _  '  thc  fl8ur-s  *  the.  lMo 


form: 


'***  °Ur  8eneral  instruction  method  takes  a 

takes  a  very  simple 


Consider  .  particular  figure  x 


(x  t  '  o  insisting  of  the  ordered  see, 

>  on  tj,e  iine  quence  of  points 


3  fo  foF 


Ln  V*>  -  r«j«a  ««,. 


♦j«> 


r  r  7~ - -  k<j 

»  tj  J*i  +k  ^  2  X:  ^  ,-) 

*k  o  »  V*  J'V*  ll* 


i^oridg  for  th.  1Wnc 

“edge-effect*"  w,  obtain  ,  priilatet  1“U"*’  The”’  &r 

clie -translate*  of  X,.  Heat  V.  ob.etv*  tlThl.T  '>r*Cl!ely 

-  -^die*  tb«  to  the  t«^y  r“Uy  «  «««* 

*1  •  1°  the  order  »  r  we  can  enianerate  the 


30  that « *  «*«  end5  up  ln 

X-J  “  «  *  I-  i.  a  «•  ula  h2J  7  WIU  f°“"d  “«  **~t  point 

2j+l  we  will  have  f0und  it„ 

its  rightmost  point  x 

p  xnc  X  .  in  either 


t  •••>■  -  -.is-,.  • 


- 


^  XL 


1  ♦ 


case  we  c*a  construct  an  appropriate  *  e*  . , 

pi  prxate  *.  Hence,  finally,  we  •#«  that  tharo 

r;; for  *n'  *tw  *»  *  — •  •* — -  ««*-. 

h'  U"‘“  °f  >0-  -  “>«.  t.  no  prohl..  .bout  ’ 

all  p-aupports  are  finite.  17,1 

7ao  Apparent  P*mA~r 

Consider  the  cm  of  2^  -  ^  [$N^j 

•  “*  3“'  •h0TO  th*t  -  •**  «a  ♦  chat  *. 

of  thi.  «gura.  Hence  d  Mat  «).«  th.  fltTOi 


* '  twa  *“  <“«>  0*  th...  figuc  have  uttb  ^  ___ 

-tupl.  distribution  apactrun  (so.  Ch.pt.rs 

—pc«T.  0.2  «nd  6.5)  „p  to  ord.r-21  u 

“  “i  “Ch  1  *«— *  **.  »  N*  t»  unit,  apart  «d  »  HU 

unit.  .part.  Thar.f,ra.  11  .U  ,roup-.,ulv.l.oc  ...  ^  fh.  . 

^  P*CC*Ptr0“  fc*  —  «  dUtln»ul,h  than.  Thu.  ,2  «  * 

apply  the  group- inv.rl.oc.  thaora.  v.  vould  in  l.cr  obtain  a  proof  that  a. 
parcoptrou  of  ord.r-2  «.  li-tloguLh  bot™.  tha...  Thi.  uould  b.  _ _ 

;  *  -  group-lnvarlanea  thaor-  *.  u 

apply  to  predicate.  Invariant  undar  infinit.  group..  Uh.„  a  ,r0Uf  „  ful.~ 

cyclic,  aa  in  the  toroidal  space*  we  h  *  **»**••  «•«•» 

apace,  we  have  considered  fro.  tioa  to  tim.  oa.  ^ 

always  use  the  group-invariance  theor—  .  . 

,  9  k#  #qu*1  th«  coafficienta  of  tulnl- 

*  S*  Butw«  us.  it  together  with  atratification  to 

on  infinite  .groups.  ’  C0,*trttC'  **  ****«•• 

-*•  Wlth  Anfinita  groups  ve  can  use  atrstifir.n  * 

we  postwar,  th.  po..ibilit  ,  *"  ku,  U*, 

possibility  of  getting  unbound*  co.ffici.nt.  vlthin  „ut,.U« 


♦  tk«  <h.  ^  ^ 

*•*»«.  tlwm  ^  w  1 

7.11  Pr^Vl^ 

A  no*.,  of  UM.  0(  ;^tl,.tlon 

b,t-n  the  «•  •*-***«-..  ^  22Z2  "uum 

d*— -  -  -  _cyeac  2  ZT,  ***** 

°f  predicate*  can  the  firo  tfc  factor*?  Far  ufcat  kiafc 

c*n  th*  *r°up-lnvarlanc*  tlwM.  k 

r predictt-  h-v-  |-*  in  .c  rr  t-  *** 

*•*  — 4.  th.  •  WW»  ,  *  "  “ 

OPPllC.tiOR  ,  ^  ******  * 


la  no  baiiiwt 


g*r  rotator 


*«>  f  <*(«> 


*wect  th“  ** «"  *  **  - - -*i. 

*p“M  •  -  -  «u  «««.  Slttutt„  22  z  m 

Remarks  y  •roM*a  laclndlaf  ratatln, 


]'  '  r"‘  1<’“‘  **"  "  *****  ««  ^UvalMct  fnUm,  llw  w 

<7-5  «*!  7.*  -r.  not  of  ,lnl„  ord„  .  ’  *“*  “**  •* 

«•  — P„  ;r~ — wu- 

hU8‘  C0*fflel*“-  *  no  f„  " 

thnt  th.  co.ffid.oe,  ,row  f,„.,  *****«»»• 

-  a  im  *• *  ^ 

sequentially  upon  tha  figur.  wlth  *°  *  *"**  *****  U#t 

* r*’  wlth  *  ••*«•«*,  Of  iroHF  trenaf nr  .. 
until  aone  apecial  «v.nf.  *  "aMfowatioa  ala mm 

peciai  event  occur.,  Mtabliahln,  it,  , 

*  t9  "*•***>  in  C  ,  a.4  fta« 

•  - 


X  o 


CHAPTER  3: 


8.0 


IH8  DIAMETER-TTMIteq  PERnRi»-ri»u, 


„dia  “e  *—  Che  -  limitations  oi  the 

meter  limited”  percept™,,  those  1„  which  4 

circumscribed  portion  of  -the  retina  R.  '  “  * 

™  -  “*■“ 

althin  .a  circumscribed  region  of  dlfaeter  lZ]Z  ^  ^  ^ 

^t  1S'  “*«“  »m  <  0.  We 

SMU  “  dimensions  of  the  space  /  “  ‘  ** 

email  enough  that  none  of  the  ,'s  e  '  “  8h°uld  b 

»•  ••••  -  ~  ~  .... — 

“ould  he  no  interesting  theory)  but  D  should  b  ”  SUUati°n’  ““  £her' 

chance  to  detect  an  i„t  82  e"°“gh  that  a  ♦.  bas  a 

an  interesting  "local  feature"  of  the  figure 

3,1  Positive;  RpQ,.1«-r 

consider  first  some  things  that  a  diameter-llinlted 

““  -  Of  the  things  it  cannot.  P°tM,,tr0n 

8,1,1  .Uniform  Picture 

A  diameter-limited  perceptron  can  tel,  n 
eatirely  white,  choose  *  ..  that  ”  "  ‘  PlCt"re  15  «•*. 

-lap,  and  define  ,  to  be  ter  L 

Then  1  °  ^  “-““d  “my  if  a„  the  points  it  can  see  are 

Picture  has  one  or  more  black  point,,  and  not  if  th  , 

ot  the  Picture  is  blank. 
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Similarly,  we  could  define  a  •  ^ 

all  others.  '  1  *  dlstl"*“1"k  “»  *‘Hleck  picture  free 

These  patterns  are  recooni,am  * 

ognlzable  because  of  their 

charaer-r  /  ,  neir  conJ unctlvely  loc'»l,r 

character  (see  Introduction):  no  d-unit  can  reati 

evidence  that  the  ft  ’  8ay  tl,at  *  etron* 

the  figure  Is  all-white  (for  there  Is  only  the  <UghtMt 

correlation  with  this) .  but  any  *  can  definitely  say  that  It  ha,  , 

evidence  that  the  picture  la  not  all  white  Soe  ’  ” 

this  character-  that  ~  ^one  Interesting  pattern,  hew 

-  ~ .....  “  ■ • ■•-  -  -  — -  -- 

cn  he  detected  by  what  hap  ,  ’  *8  - 

hy  what  happens  within  a  region  of  diameter  D. 

8,1  2  Area  Cut« 

'da  can  distinguish,  for  any  number  s,  the  class  of  figurM  _ 

ia  .raater  than  S.  To  do  this  we  define  a  a  for  h 

—  ^  hlack,  0  otherwise.  Xhen  *  P°lnt  *  *  “  ** 

T  x±  >  s 

1«  a  recognizer  for  the  class  In  question. 

8-1-3  ((on- in  terser  tine 

one  can  say  that  a  pattern  is  composed  of  -™-in„  , 

each  small  region  the  e  intersecting  line,  if,  i„ 

region,  the  pattern  contains  at  most  one  line-se^ent  if  .  „ 

each  ♦  have  value  zero  when  this  condttlon  < 

condition  Is  met,  unity  when  It  l0  not, 

I  >  0 

WU1  rejecc  811  iisures  not  in  the  class. 

8-1-4  -Triangles  anh  Rectangle. 

Wg  can  make  a  cJianictar—  t 

r  limited  perceptron  recognize  the  ft. 

8  he  f*8ures  consisting 
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o£  exactly  one  triangle  (either-  solid  or  outline)  by  the  following  trick: 

We  use  two  kinds  of  *'s:  the  first  has  weight  +1  if  its  field  contains  a 
vertex  (two  line  segments  meeting  at  an  angle),  otherwise  its  value  is  zero. 
The  second  kind,  ,  has  value  zero  if  its  field  is  blank,  or  contains  a 
line  segment,  solid  black  area,  or  a  vertex,  but  has  value  +1  if  the  field 
contains  anything  else,  including  the  end  of  a  line  segment.  Provide  enough 
of  these  t's  so  that  the  entire  retina  is  covered,  in  non-b. flapping*  fashion, 
by  both  types.  Finally  assign  weight  1  to  the  first  type  and  a  very  large 
positive  weight  W  to  those  of  the  second  type.  Then 

£  -  W£  $  <4 

.ill  be  a  specific  recognizer  for  triangles.  (It  will,  however,  accept  the 
blank, picture,  as  well).  Similarly,  by  setting  the  Vs  to  recognize  only 
right  angles,  we  can  discern  the  class  of  rectangles  with 

l  <|>i  +  WE  £  <5 

A  few  other  geometric  classes  can  be  captured  by  such  tricks,  but  they 
depend  on  curious  accidents.  A  rectangle  is  characterized  by  having  four  right 
angles,  and  none  of  the  exceptions  detected  by  the  in  |  6<3>2 

we  did  this  for  axis-parallel  rectangles:  for  others  there  are  obviously  move 
serious  resolution  and  tolerance  problems.  But  there  is  no  wsy  to  recognize 
the  squares,  even  axis-parallel,  with  diameter-limited  *'s;  the  method  of 
f  7.2.5  can't  be  so  modified. 

*  Oa.  course,  this  won't  work  when  a  vertex  occurs  tho  d  . 
suitable  overlapping,  and  assienmen-  of  lilb!  bf  h  edge  °f  *  ^suPPort.  By 
it  will  always  be  a''approxSon  of  some  s^  ’  tL?  I™  but 

of  "line  segment,1'  etc??  as  well 


S?1*5  Absolute  Tempi 

Suppose  that  one  wants  the  machine  to  recognise  exactly  a  certain  «g„re 

X°  ^  °°  0thet-  *“  **  "^er-liwited  machine  can  be  made  to  do  this  by 
partitioning  the  retina  into  regions,  and  in  eaeh  region  a  function  has  a 

value  0  if  that  part  of  the  retina  is  exactly  matched  to  the  corresponding 
Part  of  XQ,  otherwise  the  value  is  1.  Then 

£  <  1 

if  and  only  if  the  picture  is  exactly  XQ. 

Note,  however,  that  this  scheme  works  just  on  a  particular  eject  in  a 
particular  position.  It  cannot  he  generalised  to  recognise  a  particular  object 

in  an£  position,  In  fact  se  show  i„  he  next  section  that  even  the  simplest 

figure,  that  consists  of  lust  nn»  nn<e» 

J  e  point,  cannot  be  recognized  independently  of 

position! 

8-M  3-e  Figure  Containing  One  Stools  Black  Voi-e 

This  is  the  fundamental  counter-example.  We  want  a  machine 

*  0 

to  accept  figures  with  area  1,  but  reject  figures  with  area  0  or  are.  greater 
than  1. 

To  see  that  this  cannot  be  done  with  diameter-limiting,  suppose  that 

(♦*>,  V  .ed  e  have  been  selected.  Present  first  the  blank  picture.  X  . 

Then  if  f(X)*Ea.$ . (x),  we  have  f(X  )  <  b  n  ^ 

i  i  nave  1^;  <  0.  flow  present  a  flgyKe>  x  # 

containing  only  one  point,  We  must  then  have  1 
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The  change  In  the  sum  .we  be  due  to  .  cheng.  In  the  u.luce  of  of  the 

♦  In  feet.  It  oust  be  due  to  changes  only  in  for  .hid,  ,cs«>.  .i.c. 
nothing  else  in  the  picture  has  changed.  In  any  case, 

f(*l)  ~  f(X0)  >  0.  (1) 

so.  choose  another  point  a  u„ich  is  further  than  D  ana,  fro.  a.  Then  no 

sw  can  contain  both  a,  and  ag.  For  the  figure  X2  containing  only  a2  we 
oust  also  have 

f(X2)  *  io1^i  >.  e  (2) 

how  consider  the  figure  X12  containing  both  a,  and  a2  .  The  addition,  to  X2 

of  the  point  a,  can  affect  only  *•.  for  which  acS«),  and  these  are  changed 

eaactly  as  th,y  are  changed  when  the  all-blank  picture  X„  is  changed  to  the 
picture  Xj .  Therefore 


f(Xi2)  «  f(X2)  +  [f(Xl)  -  f(x0)J 

and  by  (1)  and  (2), 

f(Xj2)  >  0 
but  we  require  that 
f(Xi2)  <  0 

hesark:  Of  course,  this  is  the  sane  phenonenon  noted  in  Chapter  0.3  and  in 

Chapter  2.1.  And  it  gives  the  method  for  proof  of  the  last  statement  in 
Chapter  8.1.4. 

8*2.2  Area  Segments 

The  diameter-limited  perception  cannot  recognise  the  class  of  figure,  whose 
areas  A  lie  between  two  bounds  Aj  je  A  <  A2. 


I 
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*£oo£i  this  follows  from  the  method  of  6  8  2  1  wM,*h  * 

1  J  o.i.i,  which  Is  «  special  case  of 

this,  «ith  A,  -  1  and  A2  -  1.  (,«  ^  th,  Mthod  #f  f  ^ 
example  <vii>,  this  recognition  ^  ^  ^  f  ^  ^ 

limitation  is  relaxed). 

^•2.3  Connectedness 

The  dianeter-limited  perception  cennot  decide  .h„  th.  picture  i.  . 

T*‘  "h0le'  “  fr<“  -  or  -r.  discount  plw.. 

.  point  th.  reader  uiU  h.ve  no  dlffic,lty  in  ...ing  the  f„r»l  correct-, 
the  proof  we  gave  of  this  informally  in  Chapter  0.3. 

Proof:  consider  the  four  pictures 


Fig.  8.2.3 

:  zzrzT„  *  “  •'  — — • — ««.. 

01  io  connected,  but  XQ0  and  Xu  are  disconnected. 

Suppose  that  there  were  a  set  o,  p's  and  o',  and  ,  l(„  tor 


r  V^)  .  9 


*  W*oo>  <  9 


l  Vittu>  * 9 


l  Vi<*io)  * 9 
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•p  that  tbN.  four  figure,  were  correctly  separated.  But  then,  just  a.  1.  the 
previous  argument  we  would  have  for  all  4 

*i  <Xll)  »  <X10)  ♦  ♦1  (Xoj)  -  ♦i  (Xqq) 

**•  *"°  regions  are  wore  than  D  apart,  hence 

1  °i*i  ifl  +  e-  9-  e 
contradicting  the  separation  requirement. 

8,3  P^Wter-lImlted  Integral  Invariant 

observed  in  $  6.3.1  that  convexity  has  order  3,  but  that  the 
cosatrnctlon  UMd  tlwr.  mad  not  carry  ov.r  to  th.  dlnwt.r-U.ltad  cm,. 

b~*“*  “  "*«*  *  fi*“«  1th  t»  vid.ly  „p.t.t.d  coov„ 

0.  th.  oth«  h»d.  f  8.1.4  hou  .  di«.t.c-U«lt.d  pr.dlc.tu  can 

C*,t“"  F,rtlCUl*r  “"**  Th.  I.tt.r  construction - - 

bnt  l«d.  lot.  a.rlou.  problem  d«  toltnac.  r«aiy,  i„to  ,u„ti*  ’ 
about  differentials. 

Sup*.,  thst  w  d.flo.  •  dinwtar-lirttad  f«ol,  of  predict..  * 

**  follow  ldw=  ChooM  „  c»o.  Covor  R  with  n  partition  of  snail  coll. 

V  '°r  “Ch  lDtM*t  k  «*-  »Jk  to  h.  1  1,  Cj0*  contain.  so  ««.,.» 

change-in-direction  >kc  and  otherwise  *Jt.  •  o. 

Tjk 


f 


*•8— 


Now  consider  the  "integral* 


££  e*f 


jk 


jk 


COntrlbUtl°”  *  *“•  -  -  of  curve  Ulu  be  c.^c 

Zl  C  UZ  Cha“8e  10  dlr“tIOn  °f  •—  k»  total  .«  g. 

«  otal  curvature"  or,  rather,  the  total  (curvature  |.  finally  «  cl.u 

that  we  can  "reaH*«"  ,,  y  Aai* 

realize  \p  convex  as 

^CONVEX  *  l~ 2  e  <  2nl 
jk  Jk  1 


because  the  total  I curvatneo I  r 

I  curvature}  of  j*  figure  M<t  _  ,  ^ 

convex  figure,  achieve  the  equality  We  im  '  ,,  ’ 

the- -retina,  and  auch  matter.  ^  T  —  - 

A  sinilar  -argument  can  ha  uaed  to  conatruct  a  predicate  that  u...  th." 
slKned  curvature  to  realize  .  • 

G(X)<  n 

the  function,  of  the  Euler  characterise!,,  , 

■  J  -  •  “rl,Uc-  A1**  fMt  invariant  i,  ,«.t  tb. 

total  signed  curvature  divided  by  2ir. 

*  ,1?:  ... 

One  could  go  on  to  describe  store  aovhlm+4  .  / 
f .  P  iatlcated  predicates  that  classifv 

figurea  -by  propertie.  of  their  "differential  apectra." 

However,  we  do  not  pursue  tlvfa  >>a 

puraue  thi,  because  these  obaervation.  already  rat.. 
nuaher  of  serious  questions  about  .  T  * 

nrohi  v  and  aPPro*iaations.  There  are 

probleas  about  the  unlforalty  of  the  coverings  the  SJ  '  e 

limited  cells  c  and  problem.  ’  '  •,  °  -  ‘  *“'  dl“*t,r- 


< 


i 


c <? 


"  o- 

v  f-  ‘tetSg- 
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«koot  the  ctaulative  error.  tn 

•“*  **  **  -  * — -p  In  a, .X77  Tlu“‘ 

r  ^  -» —*• — — ;:::  ^ -u 

**  “*“*  “  '»«  underlying  «.h,  or  ^  "* 

features  of  the  x's.  °^>«red  to  the  relevant 

.jr.Tr  “■••»-.  -  _ .. 

••*.-«  P«  nrtllect  !n  thl.  content,  ~ 

***-*•  *“  deecrlptl.n  in  Chept.r  a.w  of  „  “  **  "  *»— • 

18  worded  In  such  a  way  that  ^  °f  ***  *“*  Plicate 

reasonable  size  ranges.  *  *pprox4"«tionat  within 


i 

1 


C 


CHAPTER  9:  MAGNITUDE  OF  THE  COEFFICIENTS 
9.1  Coefficients  of  the  Parity  Function  ¥ 

-  -  Wu(. 

In  §3.1  we  discussed  the  predicate  YpAR(X)  =  Tjx|  is  an  odd  ntnberl 
and  showed  that  if  §  is  the  set  of  masks  then  all  the  masks  must  appear  in 
any  L($)  expression.  One  such  expression  is 

¥par(x)  =  rs(-2)is(,1i)lni(x)  <-n 

which  contains  each  mask  with  coefficients-  that  grow  exponentially  with 
the  support-size  of  the  masks.  We  will  now  show  that  the  coefficients 
must  necessarily  grow  at  this  rate,  because  the  sign-.changing  character 
of  parity  requires  that  each  coefficient  be  large  enough  to  drown  out  the 
effects  of  the  many  coefficients  of  its  submasks.  In  effect,  we  show  that 
^PAR  can  be  realized  over  the  masks  only  by  a  stratification-like* technique* 
So  suppose  that  we  have  ~  ^*i^k  >  •  Suppose  also  that  the  group- 

invariance  theorem  has  been  applied  to  make  equal  all  a's  for  p's  of  the 
same  support-size,  and  suppose  finally  that  the  discrimination  of  y  is 
"reliable,”  e.g.,  that  ^  2  for  odd  parity  and  <  0  for  even  par¬ 

ity.  (We  use  "2"  instead  of  "1"  to  make  the  proof  slightly  neater.)  Then 
we  obtain  the  inequalities 

a:  >  2 
a2  +  20^  <  0 
a3  +  3a3  +  3al  ^  2 
•  *  ♦ 

*  But  not  by  stratification  itself,  because  the  order  cannot  be  bounced. 

In  this  Chapter  we  return  to  finite  |r|  spaces. 


'  •‘XnjfL (An 


■»r 


* 


m 


I  (?)a,  if  “1*  <x*d 

i*l  1  1  if  XX  1m  even 


Subtracting  eucceeeim  Inequelltlee,  «  d.flM 


-T^K-Ick 

“Vl  +  [  If?)'  (?)]«!  •  Vl  + 

•  I G)  V, 


*o  that  for  all  n. 


("1)n  Dn  *  2  or  [<-l)n  Dn  -  ijj  o 

Using  these  inequalities  ww  lhh  ^  a 

■  8111  ol,,:«1u  »  t-ouus  on  the  coe  flic  lento 

lOj}.  We  mu  sum  the  Ineouelltlee  mth  J 

qusxicies  with  certain  positive  weights; 

choose  any  M  >  0,  and  consider 

Th“  I  /i)  (-1)1  Bt  b  if  («)  .  zm 


-  > 
v  > 
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Ibe  left-hand  side  is 
M  i 


Jo  Jo  (’1)1  (?)  “  Jo  Jk  (“1)lak+i(^(i/ 

L  Jr  (“1)±  “k*1  fecifel)  (il(M-i)f) 


M  M 

I  l 

k-0  i-K 
M  M 


l  X'-i>1v.(*)(t d¥«bf> 

!  vi(!!)  (-i)k  (  («•*>»  \  <-»J 

k-0  *  j-0  V  j !  (M-k- j ) !  / 

1  Vi  (“)  (-1)k 


Vi  <-» 


so  we  have; 


2i$o£S:  the  ratio  f  the  largest  coefficient  to  the  analiest  coefficient 
of  any  mask  must  exceed' 


Th«ee  value,  hold  for  the  average,  ao  if  the  coefficlenta  of  each  type  ar. 
°°t  equal,  aone  must  be  even  larger  I  This  shows  that  it  is  impractical  to 


use. mask-like  *'s  to  recognize  parity-like  functions; 
afford  the  huge  number  of  *»s,  one  would  have  also  to 
of  their  coefficients! 


•even  if  one  could 
cope  with  huge  ranges 


WHanTem ».  . 


c 


*  * 
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«*•  *>•«  a  practically  f.taI  eff 

—a.  At  w  2w  lMtMcM  of  -  -  -  *  ^ 

C°  W  the  iargear  coef£Icl.nt.  *  "“l**1  Part,™  „  ra,*,., 

-  -  —a  fater^  "•  — “  -  »ra. 

fOU°“S-  «ar  the  in_Jcr  l0M"  0rd"  ■  X. 

V  °*  C0emcl—  *  Sweater  than  “*  *« 

°f  Patterns  recogni2ed  by  *  ne*ded  t0  8t°r®  the  «tire  ,.t 

Unif°nn  rePres®ntation  of  thTa  •.  ^  ^  ^  **“  8Ub8Ct8  °f  *•  *>r,  any 

there  are  2 lRl  1  8llow  /*!  b*ts  for  *  k 

2  coe^cient8  ^  total  **  "*•  .„d  .lac# 

*■  other  hand  there  flre  ^  ^  °*  «»  tequired  ls  |r|  . 

an  |Rj-bit  sequence,  so  «,ae  °f  *•  ■**  »»«.«*.«.  by 

““  3ubs«a.  ’  wouW  **««  to  repraaent 

“  Sh°Uld  “ls°  b*  -tad  that  *  ,a 

ra“'  the  —  — x  thll  Jn  »««d 

b00lM"  fu”=«""3  are  lln(S„  ehr,  .  “S  ^  311  »«.  22 1*1 

‘"formation  and  Bl11  «qoir*  2 1*1  bit 

’  a"d  -“"'““iforwty  „f  coeffiei  '  *  °f  “*««l«t- 

tWS  by  “  substantlal  factor.  *“  WUU  b‘  «P«tad  to 

w — - 3  -ctfoa  aai;, of — - - 

masks  make  rath**.  8  P1  k  ■  a  worst  $,  rn  * 

3ther  a  ®00d  b«e  because  coefficJ  *“*  th* 

06  Mrger  than  I^J  w  2IS(Ul)l  6nts  OVer  *«8ks  never  have  to 

PredICaCd  P-‘«ve  no^ZZ!'  ""  ^  ~"g  "  “«**■*» 


Let  R  be  a  aet  of  points,  y^  ....  y^,  z^,  zr  and  let  [Y^  and  {z^} 

each  be  enumerations  of  the-  subsets  of  the  y*s  and  z's  respectively. 
Then  any  figure  X  c  R  has  a  unique  decomposition  C  *  Yj  U  Z^. 


We  will  consider  the  simple  predicate  <jr, 


EQ‘ 


VYj u  V  a  rJ  - k1 


which  simply  tests,  for  any  figure  X,  whether  its  Y  and  Z  parts  have  the  sane 
positions  in  the  enumeration.  The  straightforward  geometric  example  is  that 
in  which  the  two  halves  of  R  have  the  same  form,  and  Yj^  and  Z^  are 
corresponding  sets  of  y  and  z  points. 

We  will  construct  a  very  special  set  §  of  predicates,  for  which  f  «  L(#) 
and  show  that  any  such  realization  must  involve  incredibly  large  coefficiaata! 
We  want  to  point  out  at  the  st^rt  that  the  §  we  will  use  was  designed  for 
exactly  this  purpose.  In  the  case  of  we  saw  that  coefficients  can  grow 

exponentially  with  the  size  of  | R j }  in  that  case  the  $  was  the  £«£  of  masks, 
a  natural  set,  whose  interest  exists  independently  of  this  problem.  To  show 
that  there  are  even  worse  situations  we  construct  a  $  with  no  other  interest 
than  that  it  gives  bad  coefficients. 

We  will  define  $  to  contain  two  types  of  predicates; 


^(Yj  U  Zk)  -  (i  =*  kl 


£i(Yj  u  V  “  r<J  -  kAl  m  k>vu  =  k" V  <  k>1 

each  defined  for  1*1,  ...,  2n.  Note  that  (s^)  |  =  n  and  Is^)  •  2n 


mrmmm 
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Fi'kv;t 


we  must  show  that  iL,  e  „  .  . 

sue  consider  eh.  proper  f0„, 

l*a,  ‘  l  21  <*j  -  X4)  <  i") 

Caae  I:  j  ..  *< 

IhC"  *K  -  1  xk  -  1  »•««  *tQ  -  *>ci  -  i,  ,  J  .  x 

C«e  II:  j  /  J  t  k  -  l 

Only  ^  »  i  .„d»!Q  .  Gk  J  .  0 
Case  Hi:  j  k  .  j 

non  *h(.  1 


So 


undxiil  tor  1.X . fc.j. 

kq  ■  &  -  r  a*  <n .  i;  j .  0 


«nd  the  predicate  hold.  only  for  the  j  .  (, 
is  indeed  in  !(♦). 

Now  we  establish  bounds  on  the  c 


c***»  as  it  should.  $6  + 


«Q 


^KQ 


ooeffioLn,,.  consider  any  expression 
'  *  °i  *i  +  I  »x  >  «) 


n«n  for  .eta  y^^  m  ^  <  9_ 
for  sets  Y,  Z.  we  get 

k  k  ek  9  +  i. 

«»d  £or  cat,  g« 


»1+  •••  +  Vl+  61 


<  0 


t 


i; 


§ 


I 

1 


a 


>  - 


-/- 


n-1 


We  =«n  let  by  suhtractln*  lt  f„m  ^  ^  ^  ^  § 

Appears  in  each  inequality. 

Then  8^0  and  *  1.  Then,  slnce 

ak  *  1  +  «!  +  ...  +  _x 
we  have  immediately  c<2  *2,  <*3  *4,  ...,0 

SlM‘  the  ^  1  *•  2".  <•  M.h«t  .  „„.t  *  ,t  le..t  j 2 

W“  **  Ut8e  “  «“  separation  ten.  (e  + 

r- — ~ 

that  an  expression  "i  * L n  .  .  , 

*  ,  .  1  e,UlVale"t  “  «<«  *r  V  appears  alraai, 

within  the  definitions  of  the  v  » Q  , 

^  ■  i  .  and  it  is  there  precisely  to  not-,uite- 

*  ally  weaken  their  usefulness  in  L(4>)  Thuc 

W*  lhus’  one  can  not  conclude  that 

cne  *PAR  result  is  just  due  tn  Msa. 

PAR  to  the  poor  choice  of  the  sat  of  a.*.  for  ltm 

♦  base.  (Problem:  find  a  4>  that  wa!,o0 

'  2 |R|- constant.  ““  CO,fflcl*»t*  ■>*  *PAR  gron  UR. 

«'  .  .  *  Solution  in  Chapter  9.3). 


>  ■  Ironically ,  f  ve  write  \l>  ±n  tPrmc  , 

*EQ  ln  terms  of  masks  we  have 

%  M  Q  Myj  +  Mz±  -  2Myi?i  < 

•«a  the  coefficients  are  very  small  indeed! 

Problems :  in  9.1  *  haa  olRl  , 

2 |R|  !jlK|  e-a"entS  and  *PAR  re1ulres  coefficients  like 

•  2^*gl  elements  but  the  coefficients  are  like  2R^ 

It  ia  possible  to  make  fa  with  un  to  mIrI  , 

ere  *■  a  ,  1  elements.  Does  this  mean  there 

»  s  and  i  s  with  coefficients  like  2*1  «,  thInk 


.  ...  r.tn 


t  /p 
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il*l  6  *h0Wn  th*C  °“e  nCVer  needs  co*fficl 

^  for  any  *?  w-,  *nt  ratios  larger  than 

1  nake  »ore  precise  .  ™*» 

tMi°-  C‘"  **  *  —  that  th,  hJJT  b*tVM”  — 

.  eive  oa  tte  ^  ^ 
— 1.?  c.  ycu  ^  -  — t,  « 

Predicates  in  Chapter  7?  b<>Und*  f°r  COeff*cianta  fcr  tk* 

predicate 

*EQ  •  fa*  -  Xj)  ,  7J 

ty  “oh  like  those  obtained  by  the  at 

°“t  at  each  l.vel>  t  t„  .  *  ““  „choi  . 

A*  i,  the  coefficlent  *  .  "awod,  1* 

°f  of  th.  coefficient,  of  ’  “  **>•  worse  cast 

— or  thoa?r:r 'vei- 

^  ll0Mr  for- elth  ,»ZUr  CQe  “  th*re  d<>  •*  *«*« 

(1th  respect  to  given  "  *’  “*  thl«  ««Wt,  to  a.  that 

2^  °Ut  thaC  th6re  “  *  shortage  of^  ««*• 

—  ’•  -oh  of  the.  really  *  -  -  U 

Stowth:  that  I.  t„  say,  ..  don't  hay.  .„  ‘'"“““““-Uh.  “efflclut 

»tr«tlflc,ti0„.„  7  “thod  to  detect 


> 


•f* 

> 
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9*3  ^dlC«te  With  Possibly 

11X11  "  b*  the  Ind“  °£  X  i»  «  ordering  „f  ,U  the  eub.et. 

°f  R‘  W'  °U1  the  ^  *—*•  •'I  |pak|  |  *  (Tlxll  i.  oddl 

-1th  reepect  to  the  foiling  set  0 . 4|(x||}  ot  predleates; 


f0  If  ||X||  <  i 

°i(X)  -  j  1  if  llxll  -  i 

(.  t||xl|  -  i)  nod  2  if  |  |x|  f  >  i 
Ih“  | PAR 1 1  ls  io  ««  and  is  in  fact  realized  by 

1 1 par 1 1  -  rr  c-i,i  Pi  .t 

where  F±  is  the  i-th  Fibonacci  number: 


{Fi}  "  {1'  3->  2  3>  5,  8,  13,  ...  }. 
ffisorar  eny  form  in  L«  f„r  p. .  , , 


large;  the  Fq  grow  approximately 


1 1 PAR | j  mu8t  have  coefficients  at  leaat  this 


3  J 5  +  l  n  „  0.7n 

so  that  the  largest  coefficient  is  then  like 

^  23S(  /5  +  1)  .  ,2n 


The  proof  of  the 


theori 


be  loferr*d  ‘y  » trying  the 


*****  below: 


»  can  be  seen  if 


«!  <  0  arui  amount,  are 
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conlectura 


Tiiis  predicate  and  ita  ♦  haa  the  same  quality  aa  that  in  Chapter  9.2  - 
that  the  ♦'a  theaaelvea  are  each  almost  the  desired  predicate.  Kota  also 
that  by  properly  ordering  the  subsets,  ve  can  still  choose 

*| IparJ  J  "  ♦par 

He  conjecture  that  this  example  is  a  worst  case:  to  be  precise,  if  # 
contains  |*|  elements,  the  maximal  coefficient  growth  cannot  be  faster  than 


tjL 5  -4-  1. 


where  the  exponent  constant,  is  the  Fibonacci,  or  golden  rectwgle  ratio. .. 
Our  conjecture  is  based  only  on  arguments  too  flimsy  to  write  down? 

3k*  <?F9W  iRYgyiaQce  Theorem  and  Bounded  Coefficients  On  The 
In  f.7.10  we  noted  a  counterexample  to  extending  the  group  invariance 
theorem  C$2.3)  to  infinite  retinas.  The  difficulty  came  through  using  an 

infinite  stratification  that  leads  to  unbounded  coefficients.  This  in  turn 
raises  convergence  problems  for  the  symmetric  summations  used  to  prove  equal  the 
coefficients  within  an  equivalence-class.  If  the  coefficients  are  bounded,  and 

8r0UP  C°nt£i  *  '  '  tran®l*tion  group,  we  can  prove  the  corresponding  theorem. 
We  do  not  know  stronger  results:  presumably  there  is  a  better  theorem  with 

Such  aa  the  fact  that  /5  usually  occur,  in  upper  bounds  in  the  theories 
of  rational  approximations  and  geometry  of  numbers. 
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<«)  *  condition  on  th«  co.Hici.nt,  ,nd  ,M 

condition  on  the  orounl  m.  *  ...her  atructure 

°“8roup,.  The  proof  depends  on  the  geoaetrle  fact  ah  . 
increaalng  concentric  circle  .k„  .  ttat  for 

*  *bout  two  fined  center. 


^  % 


th*  Proportion  of  area  in  co— on  „ 

■■on  approaches  unity, 


Ut  *  he  .  predicate  inyerimt  under  tr.nsl.tion  of  the  i  « 

it  the  coefficient,  of  the  ..3  are  ho  „  „  **“• 

then  there  eniot.  .n  equiv.Lnt  p  *”  MCh  equlv*1*nc.  cUm, 

equivalence  cla...  r”‘’"°n  Bltl‘  Smsi  in  e.ch 

^1:  v:r/qjysutima  Bith 

Than  define 


*C<X>  '  geV  '^f  ^  *  8>n 


rf  **>  =«  -  sel 

’  S  8  V  g 


rs-WDEo  sei 

9  g  g- 


o 

*  ***& 


t.c  U  closed  under  the  .roup  Inverse.  By  the  .rgo«nt  otf  2.3 
*<=  *“  ',“1V*1“t  “  *  “  *  Pr*dicate.  The  following  leaaa  shove  that  „ 
“lMt  "  10CrM5lOS  Cl*  C2 . >  for  -hich  the  Unit  of  the  aver... 


lim 

i  ->  oo 


hM  “*  of  ♦  Within  every  equivalence  dess. 

LEm‘:  SU,,PO*e  S°“  tunctlo“  «*>  i*  boundedjis., | f  (x)  | <  M>  ln  .2. 

Then  there  exists  s  sequence  of  increasing  radii  s„oh  that 

R  ■!".  lair  J.  f «“l  <  » 

1  * 

“““  V3lUe  ^  ^  •*  cbe  conmon  center  c,.  if  the  Unit  exist, 

for  any  center  at.  all. 

— :  Ch“3e  “  Center  the  °rl8ln  “d  «*  a*quence  of  R^s  increasing  wither 
bound.  Then  for  each  i  we  have 

hiTS^  J  *<yM*l  <  m 


“  by  coupaCtness  we  can  choose  a  convergent  subsequence;  cali  .this 
sow.  Row  given  any  other  center  x'  for  the  cirdes,  note  that 

(fc£-  I  J  f(y)dA  -  J  f(x'  +  y)  d*|  <  ) 

1  ci  Ct  2xRi  J 


{R]J. 


* 
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where  a  /x')  ,-k^ 

i  he  area  of  overlap  between  the  original  C  and  n 

centered  around  x'  n„,  .  *  Ut  mi  the  new  c! 

‘  BUt  “  Che  ««ns  grows,  for  any  *•  1 


I 


lim 
i  *f  • 


m- 


The  limit  will  exist  except  under  the  no*)  f 

««)  at  infinity.  P'CUUar  oonhitions  on  the  behavior 

To  prove  the  main  theoreu,  we  sl.ply  ch0ose  a  representati 
equivalence  class,  and  set  *  tlom  *** 

f(g)  *  o 

eewtdieg  g  as  a  translation  fro.  the  origin. 

“  fOUO”  ^  ““  Perceptron  obtained  inf  74Math 

coefficients,  and  that  there  i,  „  '  “"bounded 

here  is  no  equivalent  member  of  i  .a 
coefficients.  with  bounded 

Note:  the  methods  of/9.2  and /9  3  are  .. 

(19411*  t0  r  t0  thoae  “,d  hy  Mghill  and  Kauts 

1  fl“d  coefficients  for  the  orcer-1  case.  Th.  „ 

integer  coefficients  «-»,  ^  show  that  with 

coetticients  there  is  an  order  1  predicate  for  uh*  k 

exceeds  -2-i  .  2n  hich  SOme  coefficient 

e  n  • 


*>■■ 

v> 


*  Mghlll,  j.  and  Kautz.  u  h  •»/>„ 

Switching  functions,..  IRE  on  llttt'o.lc^”™ 


J  '■ 

i 


CTAPm  10:  LtABMTMr 
10.0  Introduction 

S-PPO..  on.  uaht.  .  Machine  that  "dlacrlmlnat..” 


between  two  ^eta 


tP)  -  Pj,  ....  p^  noj  £q)  ,  ^ . 


•t  figure#.  Assuming  rhat  .  net  {  o£  ptedtcates  t,  avalu.JU>  ^  ^ 

find  rh.  coefflelenca  of  .  function  ^  ln  l(#)  with  ^ 

every  k, 


Vq  (pk}  *  1  and  ♦pq  (Qk)  =  o. 

Ibat  is,  we  would  like  to  find  a  set  fn  }  , 

f  *-aqr  coefficients  such  that 

r%^)  >  01  but  rz  <  0-j# 


But  suppose  further  that  for  acne  reason  we  don't  want  to  de#ign  ^ 
••chine  especially  for  this  job,  perhaps 

•'  1>-  *SUWV^f|}^I0“  th*  ~dtl“  before  we  are  told 

U>  b*C*U“e  Ch*  **  -»  changed  at  a  l.ter  tlne,  or  m„ 

iii)  because  we  don’t  have  a  , 

of  [Pj  and  (q),  hence  cw™  ch^l2  fana  ric”1  d'flaltlon 

■  calculate  the  foT  *  th“r«lc»l  ™y  to 

■  ■  Then  It  becomes  tempting  to  con.lder  building  .  Mchlna  that  , 

•oc.pt  Information  end  cnlcnlnte  epproprtete  ect  of  coefficient.-,. 
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•  machine  that  "learns." 

^  th'  **"*  £~  '“tl0,“  °£  thla  <*•**«  -«  -HI  th,  theory  Qf 

.  Particularly  .1^1.  end  .legw  learning  „chlne  ^  ^  ^ 

«0«n  it  i.  given  a  .eguence  of  P-.  ..  ^  ^  ^  cU<<  ^  u  in 

“  “  JU,t  ab0Utthe^PleSt  «* *«  ^  «*«  -  «!<.  to  h.  to  n^,, 

•ud  further  on  »e  will  discn..  it,  efficiency,  end  reng.  of  cepehuir,  in 
-relation  to  ace  .ore  ,„phisticated  concept,  of  -i**^  „chl„M  „ 

-  **.»..  we  ere  now  concerned  »or.  with  th.  ,.t.  of  coefficient.  fa  J 

than  with  the  nature  of  I  it.elf,  it  will  convenient  to  think  of  the’ 
function,  in  «„  a.  a-aoci.ted  with  the  ..t.  f  }  regard.  ..  vector., 

-  we  will  oak.  heavy  of  the  g^etcy  of  the  vector-pace  who.,  h... 
vector,  are  the  *•,  in  #.  and  with  coefficient.  usually  the  Integer.. 

Warning:  -the  vector-pace  ha.e  i.  the  set  of  th:  point,  of  * 

’  in  thi.  chapter  we  will  think  of  the  for..  ^  ?  . 

iYi  eiments  of  a  v actor 

t7e;  “  ,h°"ld  *“  -  -  -  fa  ian’t  .  vector  Cod 

that  for  eech  *!(*)  there  are  nany  a-vectora*.  (In  fact,  though  it  i.  not 

XTroTlSrrt-  ah.pt-  of  thi. 

mathematical  tool  (followedUosely  by  aStiaUcs^JfT*?*  i#  th#  chl*f 
role  in  our  development.)  if  we  wLe^n  f  Which  *1*°  Pl*y»  «  Mali 
little  vaa  learned  about  percenti-ono  i  °^“nteer  one  chief  reeeon  why  so 
we  would  point  toward  the  use  of  thn  °  *he  decade  tfut  they  have  bean  studied 
the  Bptat',  a.  vector.,  the  reuti^,  w°r  *>r  in  thjnfcjg ’ 

predicate,  in  l(i)  havi  i£o£  y'£ ”  *£“»  P“t*rM  [x) 

operator,  on  the  pattern,  thec.elvcs  'h””,-  --'Ktcr.  ,r.  not  Maw 
operate  on  .pace,  of  functional op«itoJj tgf '°perator*-  th«  the, 
f-cla.ses— of  their  vector  apace,  ate  artf  Jr.  h  >’*tt«rn.-  Since  tbe  bau... 
to  discover  much  about  the  kind*  6  a^itrary»  one  can't  hope  to  use  thM 
important  question^r^t  a£ut  the^LT*8  that  wULl  lie  **  « 
tlW-orders  of  complexities  in  llne*r  Properties  of  the  L(*)'a,  but  aWe 

£ ■?*><*«> 1  •“  S.S?  1,1  “"“’““"S  qualities  fro.  thi  lifo^e^T 


h,re' th* Mt  of  *“• {o)  th,t  d,fiM  *  * — — 

;t~" . . — - — — — 

with  these  warnines  in  alnrf  -  w* 

'  *"  r*8,rd  «y  ««»«  c  «b.M>  x  of  x 

determining  a  vector  Vj  vith  componenta 

<P200.  ....  cp^fxj). 


tol  *"y  pr*dlc««  *  i«  K«)  1.  determined  by 


components 


«t  least  sons  vector  with 


^1  *  ®2»  •••»  Q  ) 


•O  that  we  can  write  an  inner-product 


expression  for  f(X): 


?A-e  V  "  Ve't  »• 


♦(X)  «->  A+  •  Vx  >  0. 


(It  ie:  convenient  to  a*<um*  t-k.*  * 

tlUl  ,  th“  *  W“i"  th*  identity  function  «Ml 

freehold  «  in  our  fomulae.) 

•-8w-geel  ie  to  find  a  diecriminating  function  *- 

i-Vacr„r-,  .  -  'pq-  or-  *1»lvalantly  an 

with  the  property 


*  VPk  >  0  and  *PQ  *  <.  0. 


For  want  of  a  better  idea,  it 
ograa,"  as  follows; 


occur,  to  u.  to  try  to  find  by  a 


ifgT 


n 
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Step  0: 


Step  it 
Step  2*. 


Set  A  ■  (l,  l,  . ...  i),  or  to  any  other  initial 
value  you  pleeae.' 

Choose  an  element  of  {p^}  or  [<^},  call  it  V. 

Compute  the  «ign  of  A  •  v.  If  it  i.  correct  (i.a.. 
has  the  proper  sign)  go* back  to  step  1.  otherwise 
replace  A  by  A  i  v  where  the  sign  is  the  one  A-V 
should  have,  and  go  back  to  step  1. 

“*  “**  11  Sl"ple;  lf  A'V  ls  l«g..  th.  chang*  will  result  tr 

(*  V>.v  -  A.v  -  |V|  next  tine,  and  thl.  ha.  a  better  chanca  to  ha  nagattm. 
Conaaraaly,  1£  A-V  i.  too  negative,  (A  +  V) •  V  "  A.v  +  |v|2  1.  mn  Ult.ly  „ 

h.  positive.  Thus  we  have  a  simple  kind  of  "f..db.ok..-Hh«r,.r  th.  eyetm 
~k«.  an  error,  then  A  1.  "r.lnforc«d»~that  la,  .lightly  «,dlflad~l„  a 
direction  designed  to  correct  the  error’. 

““  “  “°rk?  “  **“8  •‘-Pl-l-dad.  because  aach  detraction 

1.  performed  with  j„.t  one  P,  or  %  in  vie..  Why  „uld  on.  ..pact  ,met.l 

iaiprovanent  «hen  a  correction  deigns  to  correct  for  on.  P  or  Q  may  „kt  A 

«ong  for  many  others!  Indeed,  this  will  happen,  ..eecl.U,  .t  th.  bnlamlu. 

Of  the  process.  The  remarkable  fact  1.  that  thl.  procadur.  .111  ultUat.ly 

"T  ~ 'htre  e,tStf  >ny  *P5-A1AU  then  <h.  pmcdnra  ui,  1  . . 

~  •  (Then  it  Hill  continue  to  bo  correct,  so  .m  r«.t„  in  (l) 

And  there  Is  no  constraint  on  the  choice  In  Step  (1)  0n  th.  Vs,  aav,  that 
* 

That  it  finds  its  own  solution  rather  than  m,. 
make  one  think,  according  to  one's  Stitud?  °ne  W  had  in  ffilnd 
random  or  more  original  than  they  seemed  *  a  ££  percePtron«  «•  either  more 
If  there  is  a  solution  cone  (and  ther^l;  i  be“*r  ,5at«*«nt  be: 
will  find  some  point  .near  its  boundary.  "°*t  °n*^  th#n  th*  P«rc«ptw>o 


Cl! 


-•*.  h 


o  • 


•5- 


. ~ 

“  th*  Cel,br*CM  "»«=.pt,on  Convergence  lh«t«.„  . 

«»t  conjectured  by  to.Mkl.tt. 

.  **  W1U  ^  th*  th*r«  Pir.t  we  must  nelte  a  tw  eti  , 

•bout  the  geometry.  Actordine  *  •Mpulition. 

According  to  our  convention.,  the  co.poh.nt.  ,u 
A*vec tore'  .«  reel  number—th.  ..  f»n*"t.  of  th* 

to  be  integer*  without  4  *  ^  C°Uld  b*  •M,*wd 

**ri  without  loss  of  8en«riiuu  \  ~ 

“  °ni3' th*  °  - 1  •*  -  ^t  tb.~:?M  ,bov* 

re  jtrtet-  *•*  -  e  1  noc  ‘•••on  to  to 

h‘'“—  -  *-i **• - 

Define,  for  1  sk  *p 


end,  for 


fk  ■  . w, 

P+1S  k  JSp  +  q 


("9. 


l(Vp>’ 


-  9, 


(<Jk-p>’ 


■  vw>- 


The  negative* 

a  8*t  fx, 


th'  '!'V•Ctor•  r*i"«  the  problem  to  the 
fk’  Vo  •*  vector,  find  .  A-v.ctor  A 


•implese  fom* 
j  for  which 


liven 


V  £k  >  o 
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for  .11  k.  The  dlff,r.nc«  b««.n  th.  P 

The  convergence  ch»r«  will  follow 
simple  Lemmas: 


*  an<1  Q'»  hee  been  removed. 

«•  en  e«sy  consequence  of  three 


10 *1  Geometric 


Lemma  1: 
a  bound  b 


Let  Vj, 


•  •  • » 


V 


•  4  4 


on  their  lengths: 


b*  *  *,<,u,nce  of  «««■•  .uoh  th.t  ch.r.  1. 


(i)  IvJ  <  b 

«nd  for  ell  i 

Vi  *  Si  £  0 

where  St  Is  define<ll  to  be  the  vector  sum 


Then 

for  every  n. 
Proof; 


si  -  \  +  v2  +  ...  +  vt-1. 

|sn!  <  b/n 

I  SL  +  Vtl2 

lsil2  +  2St-Vt  +  IvJ2 
<  ls^|2  +  2*0  +  b2 


0  «wl  Uj,  lienee,,  by  induction,  (sj2  <  n  b 2 


Q.E.d. 


ally  this  is  very  simple.  The  condition,  i)  u )  ,tate  th-t 

Sl+1  ”USt  Ue  in  the  ,h*ded  •—*»«•  *■  W.  Then  rig.  (b)  ^Vi 
the  extreme  cese  In  which  the  hound  |s J  -  h/h  could  he  obtained. 


Lemma  2:  T.pt- 

—  1-  vn  ...  he  man*.  ..quenC!  o£  vector<  ^ 

tor  some  fixed  vector  A  Mm  » 

Inner  products  ere  hcunded  away  £rom  0: 

o  <  d  <  v  •  a. 

Then  there  is  a  constant  c  such  th^t,  £„r  .U  „ 


lvj  +  . . .  +  v  | 


>  C  • 


i.  e. ,  the  sum  grows  at  least  linearly  with 

E£S«f:  (V  +  ...  +v)  .  A 

1  "  Vi  A+  •••  +  hy  i) 


(vt  + 


•  •  • 


+  vn)-A  s  {v  + 


t  *  « 


vJ  *  Ia! 


Ceuchy- Schwartz  ineqnality.  Hence 


by  the 


K  +  -+Vn|>n.,i|. 


Here  the 


Q.E.D. 


geometry  ie  trivial;  the  projection  of  each  v  in  th  * 

»uet  exceed  d,  ao  their  ”  1  1  0,1  A-diraction 

"  their  eom  muSt  groK  «  ^  ^ 


iaaii;  Ho  infinite  se,n<mM 
— -  «.  —  I  end  hL  C“  fM  SU  “  - 

^  ""  WUW  ~  —  b  end  c  for  which 


•  n  <  b  /n 


for  arbitrarily,  large  n 


10*2  The  Convergence  Theory 


The  radial  increase  must  be  at  least  d/|A| 
in  araoimt,  yet  the  new  vector  aust  rmain 
in  the  shaded  region;  *-his  becomes 
impossible  when  the  region,  whose  thick* 
ness  varies  inversely  with  R,  becones 
thinner  than  d/ltf 


Let  F  -  fi,  ....  fn  be  a  finite  collection  of  vectors  and  that  there 


exists  a  vector  A  such  that  for  all  i 


'  A  >  0 

1 

Since  F  is  finite  there  must  exist 


a  masher  d  such  that  for  all  i 


The  learning  proceas  ia  deacribed  by  the  following  Program: 

Ot  Set  J  to  1.  Set  S^  to  0. 

IS  Choose  an  element  of  F  and  call  it  f. 

2:  If  f-Sj  >0,  go  back  to  Step  1.  (otherwise  f.S^  so.) 

3:  Define  V  to  be  f,  and  define  S  to  be  S  +  v 

J+l  j  j* 

Replace  j  by  j  +  1  and  go  to  Step  l. 

10'2!  Ihe  \ . Y  •••  Produced  by  the  Program  c.nnot 

be  infinite. 

&oot:  H.e  Cvj}  satisfy  the  conditions  of  Lmmss.  i  end  2,  so  LMH  3  appHes. 
&roUaH,:  The  nusber  ot  errors  the  Program  can  ever  aake  is  bounded  >bovc 
by  a—  3):  c-n  <  b^nd  by  ttecm.  1),  ^  n  <  w  |fJ  /b>  „ 


n  s 


lAj2  max  jfj2 

nin  I^-aJ2 


The  point  is  that  once  the  program  can  make  no  more  errors,  it  must 
have  "converged”  to  a  solution  vector.  (This  might  seem  to  be  a  peculiar 
argmnent-like  trusting  a  surgeon  because  he  has  attained  hi«  full  year’s 
mortality  in  the  first  month-but  some  reflection  should  show  the  difference.) 

10,3  Application: _ Learning  the  Paritv  Vimrt-i™  * 

"Ll  1  ■"  -  .  1 ‘^PAR 

Consider  the  problem  of  reinforcement-learning  the  coefficients  (over 

the  set  i  of  masks)  for  tpAR  as  described  in  §9.1.  The  coefficient  vector 
must  grow  to  length 


Is/  -  22b+(  J )  22<-l>+  ...  +  (;)  x *  .  S  (  »  ),2t 

J_  V  1  / 


(1  +  2)  »  5n 


<Qd  the  *  vector  has  length 


2 

lSJ  <  *  *  |X{2 


T  > 


(I)” 


(Lower  bound). 


ThUS  Le"“4  1  can  «sed  to  give  a  lower  estiaat*  t 

t  estiaate  on  learning  ti»e  (provided 

on!:^w?,a  Hiniaal  length  of  a  solution  coefficient  vector) 

-  ^fortunately,  ve  hnov  little  afcut  the  real  learning  tin...  For  , 
since  the  largest  coefficient  is  £  2n  »,  .  fp**’ 

•  *  2  »«  «lr “fy  koow  it  require,  a  2"  .  (  i  f 

"7^“'  1-‘-  2  ^  —  ««  mat.  the  largaat  ' 

^  ^  on.  pattern  X  -  g.  If  w. 

“  ^  ^  ^  *“  *-«.  «...  or  .elected  tha. 

uniformly  at  random,  this  would  average  . .  one  of  every  2n  trial,  to 

«  training  aequenoe  should  to  per.i.t  for  2*»  trial..  Even  thl.  t< 


hopelessly  optlm.tic,  for  the 

*  “  ^  in  the  negative  directi  ^  *U  ^  t0  b*  to 

8*  xve  direction,  end  ve  h»n— 

f°r  W  «  1*1  ■.!.«  lMM  23n  '  thW  th*  ,Mral”«  Urn 

*  «y  «...  u  M  '  “  •  °°  SOOd  ■“**  *»  tiMiM 

«  „  8  to  try  to  "learn"  t 

tUMly’  U.I  ten*  th.t  ^  Wth 


5n  n 

_ ;  2 

I  (Copper  bound) 


*°  we  have  the  number  of 


pinched  between 


SE2£»  and  corrections 


°*de  while  learning 


1  n 

(2P  <  T  <  (10)n 


Supp°*e  ""“v  1*  learned  e^ehov  _ 
reinforcement*.  ihls  „„  c<Me  „  *  I*r*ett  "»k  *111  get  2» 

“  '  l'  Ih'«  “»  *•  corrected  only  r.inT  *T*  « 

zrts.  r,- ;  j  ~r  ^ -rr  j  -  - 

’  “ !  y  ,“r  -  < ;  )'■-■  »•  -irsr  *  - 


;|V.*0  W  7  ,  e*  Xhi8  esti»ate  lead,  to 

(°x*)2  +(?)(f)2n+(f  Xf)V+....(|)v 


-n 

■5  reinforc 


But  this  my  be  an  underestw.  v 

^ th*  BMhi“  -  - — fw 


-  Sc  i 


-- 


* 


f  • 


— pter  a  m*m  amsm  m  m m 

U‘°  ^SSPectivitv  and  serial 

u  seems  intuitively  dear  that  the  reason  that  the  ahstraet  quality 
of  connectivity  cannot  he  captured  hy  a  machine  o,  finite  order  is  that 
it  has  an  inherently  serial  character;  one  cannot  conclude  that  a  figure 
is  connected  by  any  staple  order-independent  combination  of  staple  tests. 
The  same  is  true  for  the  much  stapler  property  of  gartay.  In  the  c„e 

of  parity,  there  is  a  stark  contrast  between  our  "worst  possible"  result 

for  finite-order  machines  (  »  ,  and  the  following  "bast  possible"  result 
for  the  serial  computation  of  parity.  Let  r, .  Xj . *  be  a„y 

enumeration  of  the  points  of  R  and  consider  the  foUodng"^^  for 
determining  the  parity  of  |xf : 

START:  set  i  to  0 
EVEN:  add  1  to  i 

*  *» 

> .  If  i  *  |r|.  then  STOP;  Parity  is  EVEN 

....  •  If  x^j.  -  0,  go  to  EVEN;  otherwise  go  to  ODD: 

ODD:  add  1  to  i 

Hi-  | R |  then  STOP;  parity  is  ODD 

Xi  go  to  ODD;  otherwise  go  to  EVEN: 

'  #here  "8°  “  °"  mMnS  CO,,,:1"ue  «>e  algorithm  at  the  instruction 
.  whose  name  is  a. 

Now  this  program  is  'Vintaal-  i„  two  respects:  fitst  i„  the  number  of 
computation-steps  per  point,  hut  more  significant,  in  the  fact  that  the 
-program  retires  no  temporary  storage-place  for  partial  information 
accumulated  during  the  computation,  other  than  that  required  for  the 


< 


enumeration  variable  i.  (In  a  sense,  the  process  requires  one  binary- 

digit  of  current  information,  but  this  can  be  absorbed  [as  above]  into  the 
algorithm-structure) . 

This  suggests  that  it  might  be  illuminating  to  ask  for  connectivity: 
how  much  storage  is  required  by  the  best  serial  algorithm?  The 
as  shown  below,  is  that  it  requires  no  more  than  about  2  times  that  for 
storing  the  enumeration  variable  alone!  To  study  this  problem  it  seems 
that  the  Turing  Machine  framework  is  the  simplest  and  most  natural,  because 
of  its  simple  uniform  way  of  handling  information  storage. 


11. 1  A  Serial  Algorithm  for  Connectivity 

Connectivity  of  a  geometric  figure  X  is  characterized  by  the  fact 
that  between  any  path  (p,q)  of  points  of  X  there  is  a  path  that  lies 
entirely  in  X.  An  equivalent  definition,  using  the  enumeration  x  , 

j  9  9  f  R  I 

of  the  points  of  R  is:  X  is  connected  when  each  point  xi  in  X,  except  the  first 
point  in  X  has  a  path  to  some  X.  in  X  for  which  i>J.  (p„of:  „y 

then,  each  point  of  X  is  connected  to  the  first  point  in  X.)  U.ing  this 
definition  of  connectivity  we  can  describe  a  beautiful  algorithm  to  teat 

whether  X  is  connected.  We  will  consider  only  figures  that  are  - - -w.j. 

regular"— to  be  precise,  we  suppose  chat  X  is  bounded  by  a  number  of 
oriented,  simple,  closed  curves  so  that  for  each  point  x.  on  a  boundary 
there  is  defined  a  unique  "next  point"  x£,  on  that  boundary.  We  choose 

V  to  be -the  boundary  point  to  the  left  of  when  facing  the  — f _ - 

of  X.  We  will  also  assume  that  points  x±  and  xJ+1  that  are - tlve 
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in  the  enumeration  are  adjacent  In  R.  Finally.  we  will  assume  that  X 
does  not  touch  the  edges  of  the  space  R 

Set  i  to  0  and  go  to 

a4££l<:  Add  1  to  1.  If  1  .  |R|,  stop  and  print  "X  1.  Nlnj.". 

If  xiex  then  go  to  SgtfJ,  otherwise  go  to 

SSffi:  Add  1  to  i.  If  1  -  |r|.  Stop  and  ptlnt  „x  , - „ 

If  xi-lelt  or  Xi*x  8°  to  SCAN,  otherwise 

Set  j  to  i  and  go  to  TRACE. 

TRACE :  Set  j  to  j* 

If  J-i,  Stop  and  print  ,»X  is  disconnect" 

If  j>i,  go  to  TRACE. 

If  j<i>  go  to  SCAN. 


,.:,Sotlce  that  at  any  point  In  the  computation,  It  Is  necessary  to  heap 
track  of  the  Indexes  of  Just  the  two  points  x±  and  x 

Analysis 

SB8S"  simply  finds  the  first  point  of  X  in  the  enu.er.tlon  of 
once  such  a  point  of  X  is  found,  SC*,  sesrches  through  all  „f  R, 
eventually  testing  every  point  of  X.  The  current  point,  x^  of  ^ 
is  tested  as  follows,  If  *±  is  „ot  m  x,  the„  n0  ^  ^  nece,e<ry  >nd 

mt  goes  on  to  xJ+1.  If  the  previous  point  ^  was  In  X  (and,  by 
induction,  is  presumed  to  have  passed  the  test)  then  x^  if  i„  x,  is 
connected  to  ^  by  adjacency.  Finally,  if  x^X  and  ^  then 

is  on  a  boundary  curve  8.  circumnavigates  this  boundary  curve. 
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r  7 is  a  ^  “  - — «> . ^.ry 

°“Sly  -*»— *  of  X,  1„  ehich  CJM 

“*  be“  — ~  before  or  «f)  ,  ls  .  “  ^  *  * 

ourve  to  lnt‘rl0T  boundary 

reaching  *  J7/  ^  ^  ‘  be“  ~ 

curve  of  e^er-hefel^  “  "  ^  B  "  ^  ™  , 

encountered  component  of  X,  the  only  cnee  to 

TRACE  Will  return  e  7  C***  in 

«-  return  to  x,  without  .eetto8  an  *  f„r  *,*„  J<t> 

*“  1  "*  “P  “  *'*'  “  -  if  X  hee  a  aingi.  ' 

connected  component.  *"**  ^ 

»ote  that  we  can  count  the  number  of  component,  0f  X  h  « 

K.  tolttolly  zero,  and  adding  1  to  K  ea  h  7  “Ct“* 

exit  K„t.  ,  each  time  ^  re.ch«. 

"  al“  th3t  algorithm  to  ,„ite  efficient,  the  , 
examined  more  than  nno  *  th  0nly  P0*0** 

than  once  are  some  of  the  boundary  point.  , 

them  are  examln«a  Y  P  tot8»  and  none  of 

are  examined  more  than  three  times. 


J* 
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11.2 


The  Turing  Machine  Version  nf  *.u  „ 

~  - - L.the  Connectivity  Algorjjjim 

IC  18  COnvenlent  to  assume  that  R  is  a  2n  x  2n  so 

x  2  8<iu«u  array.  Ut 

22n  be  an  enumeration  of  the  points  of  R  ln  the  or,„ 

1  2*+1  •  •  •  (2n-2)2n+1 

2  2°+2  *  •  •  (2n.l)2n+2 


V 


<*/■**' 

>i. 


2*2° 


2nc2n 


T  Ch0l“  "  -  —»«-  —  available  a  ^  wy 

must  1  thC  SltUatl°n  t0  3  MaChine*  The  Maehla. 

be  able  to  specify  a  point  x  of  R 

i  ^  tlnd  whether  x.  eX  < _ 

case  x  is  a  boundary  point  0f  V  fu  v  *  ’ 

°f  *•  “nd  thc  index  j*  of  a, 

neighbor"  of  x  m.  t  .  '' 

1  g  HachI“  taP*  **-11  have  the  fBta. 


®  *t  * 


33 


“h6the  de”°“'  "*  -  -  «-*  agaaree.  tb. 

;r rl8ht  °f " aod  c,n  h°u  the  *  -  *  -  *  ,i. 

*',U  SUPP°Se  Ch8t  the  ^  h-ith  th.  o.UM. 

•  «sur.  X,  thro„8h'8„  "or8cl8„  ^  ^  _ 

certain  internal  states  of  the  machine  have  rh 

the  resultin  ~  Property  that  when 

the  resulting  next  state  depends  on  whether  h. 

nether  the  coordinates  in  th* 
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X  (or  J)  Intervals  do.lgn.te  a  joint  In  X.  It  nan  bn  verified,  though 
the  details  are  tedious,  that  all  the  operations  desorlbe.1  in  the  algorithm 
oan  be  performed  by  a  fired  Turing  maehlne  that  uses  no  tape  square, 
other  than  those  in  the  intervals.  For  example,  "1-|*|"  If  and 

only  if  there  are  all  seres  in  the  "s  following  I_  and  I  .  "Add 

1  to  1"  is  equivalent  tot  "start  at  and  move  left,  changing  r, 

to  O's  until  a  0  is  encountered  and  changed  to  1  or  until  I  1,  «t.  The 
only  non-trivlal  operation  is  eonputing  j*  given  J.  But  this  require, 
only  examining  the  neighbors  of  Xj,  and  that  is  done  by  addin,  >1  to  the 
Jx  and  Jy  coordinates,  and  consulting  the  oracle. 

Since  the  Turing  machine  can  keep  track  of  which  interval 

it  is  in,  we  really  need  only  one  symbol  for  punctuation,  so  rh  Turing 
machine  can  be  a  3-syrabol  machine.  By  using  a  block  encoding,  one  can 
use  a  2-symbol  machine,  and,  omitting  details,  we  obtain  the  result:  ' 


^m^there  .ls  a  2~svmbol  T.,-^  ^hine  that  can  wr,fy 
the, connectivity  of  a  figure  x  on  any  rectangular  arrag  R, 
«gjaL.l.eaa  than  (2-fe)  1oK„fRj.  squares  of  ,npp 

For  there  is  a  similar  procedure  that  makes  three;  tests:  • 

i.  X  is  not  disconnected  by  any  vertical  line  that  does 
not  intersect  X. 

ii.  The  Intersection  of  X  with  any  vertical  line  is  a 
connected  segment. 

iii.  The  outer  boundary  of  X  does  not  change  the  sign  of 
its  curvature. 
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i 

1 1 

'  A  detailed  construction  shows  that  each  test  requires  only  one  index 

point,  so  that 

Theorem:  For  any  e  there  is  a  2-symbol  Turit*  machine  that  can. verify 
.  the  convexity  of  a  figure  X  on  any  rectangular  array  ».  using 
less  than  (1-fe)  log^  lR|  squares  of  tape. 

This  last  result  is  certainly  minimal  sir.ce  log2  R  squares  are 
needed  just  to  indicate  a  point  of  R,,  and  ali  points  must  be  examined* 
We  are  quite  sure  that  the  connectivity  algorithm  is  minimal,  also,  in 
its  use  of  tape,  but  we  have  no  proof.  In  fact,  we  do  not  know  any 
method,  in  {general,  to  show  that  an  algorithm  is  minimal  in  storage, 
except  when  information-theoretic  arguments  Can  be  used.  Incidentally 
it  is  not  hard  to  show  that  T|x|  is  prime)  requires  no  more  than 
(2+e)log2 |r|  squares  (and  presumably  needs  more  than  (2-e)log2|R|). 

We  do  not  definitely  know  any  geometric  predicates  that  require 
1  higher  orders  of  storage,  but  we  suspect  thet  in  an  appropriate  sense, 

the  topological  equivalence  of  two  figures  (q.*gV,  two  components  of  X) 
requires  something  more  like  |rJ than  like  log-W  squares.  There  are,  of 
course,  recursive  function-theoretic  predicates  that  require  arbitrarily 
high,  indeed  non-computable,  orders  of  storage,  but  none  of  these  are 
known  to  have  straightforward  geometric  interpretations.  *' 


% 


